Explicitly Modeling the Discriminability for Instance-Aware Visual Object Tracking

Mengmeng Wang, Xiaoqian Yang, Yong LiuarXiv 2021 · OpenReview

TL;DR

This paper introduces the Instance-Aware Tracker (IAT), a novel approach to visual object tracking that explicitly models the "discriminability" of target features. By framing tracking as an instance-level classification problem using contrastive learning, IAT significantly improves robustness against distractors and occlusions, achieving state-of-the-art results on multiple benchmarks while maintaining real-time performance.

What Problem Does This Paper Address?

Visual object tracking has seen remarkable progress, but persistent challenges remain, particularly in handling distractors and occlusions. The authors hypothesize that current methods, while good at learning expressive features, fall short in making these features truly discriminative. This means the learned representations might not be distinct enough to reliably distinguish the target object from similar-looking distractors or from the background, especially under challenging conditions. They aim to bridge this gap by explicitly enhancing the discriminative power of feature representations.

Methods

The core innovation is the Instance-Aware Tracker (IAT), which augments a standard tracking pipeline with an instance classification branch. This branch leverages self-supervised contrastive learning to train the model to distinguish the target instance from a large pool of negative samples.

Key components include:

  • Instance-Level Classification: Instead of just learning to recognize a target, the model learns to classify each target as a unique "instance" against many others.
  • Contrastive Learning: This technique trains the model to pull similar samples (positive pairs) closer in feature space and push dissimilar samples (negative pairs) further apart.
  • Memory Bank: A large memory bank stores features from historical frames (acting as negative samples) to provide a rich and diverse set of distractors for contrastive learning.
  • Negative Sample Selection: The strategy involves using frames from the same video as positive samples and frames from different videos as negative samples, ensuring both intra-class (within a video) and inter-class (across videos) discriminability.
  • Two Variants:
    • IAT-V (Video-Level): Treats each entire video as an instance, enhancing the ability to distinguish the target from diverse backgrounds.
  • IAT-O (Object-Level): Treats each target bounding box as an instance, focusing on differentiating between similar objects and mitigating the target-distractor dilemma.

The instance classification branch is only active during training, meaning it adds no computational overhead during inference.

Datasets

The paper evaluates its method on a comprehensive set of 8 benchmark datasets:

  • OTB50
  • OTB100
  • GOT-10k
  • LaSOT
  • NFS (Need for Speed)
  • UAV123
  • TrackingNet
  • VOT2019

Training was performed using splits from GOT-10k, LaSOT, TrackingNet, and COCO.

Key Insights and Ideas

  • Discriminability is Key: The paper strongly argues that simply learning expressive features isn't enough for robust tracking; explicit modeling of discriminability is crucial, especially for handling distractors.
  • Contrastive Learning for Tracking: Applying contrastive learning, typically used for representation learning, to the tracking problem is a novel idea that proves highly effective.
  • Instance Granularity Matters: Differentiating between video-level and object-level instance representations shows that both can be beneficial, with object-level offering finer-grained discrimination for target-distractor scenarios.
  • Training-Only Enhancement: The fact that the discriminative branch is only used during training, without impacting inference speed, is a significant practical advantage.

Key Results

  • State-of-the-Art Performance: Both IAT-V and IAT-O achieve leading results across 8 benchmark datasets, often outperforming previous state-of-the-art methods by significant margins.
  • Improved Robustness: The method demonstrates superior performance in challenging scenarios like distractors and occlusions, as evidenced by qualitative results and performance on specific attributes (e.g., scale variation, occlusion, fast motion).
  • Real-time Speed: The trackers achieve an average speed of over 30 FPS on a single GPU, making them practical for real-world applications.
  • IAT-O Superiority: IAT-O generally performs better than IAT-V, particularly on datasets with many similar distractors, confirming its effectiveness in fine-grained discrimination. For example, on GOT-10k, IAT-O outperforms the baseline PrDiMP by 5.1% on AO.

Limitations and Future Work

  • Computational Cost during Training: While inference is fast, the contrastive learning approach with a large memory bank likely increases training time and memory requirements.
  • Future Work: The authors suggest that the architectures and ideas presented could be extended to other related tasks, such as video object segmentation. They also leave open the possibility of further exploring different instance granularities or more advanced contrastive learning techniques.

Summary updated

Abstract

Visual object tracking performance has been dramatically improved in recent years, but some severe challenges remain open, like distractors and occlusions. We suspect the reason is that the feature representations of the tracking targets are only expressively learned but not fully discriminatively modeled. In this paper, we propose a novel Instance-Aware Tracker (IAT) to explicitly excavate the discriminability of feature representations, which improves the classical visual tracking pipeline with an instance-level classifier. First, we introduce a contrastive learning mechanism to formulate the classification task, ensuring that every training sample could be uniquely modeled and be highly distinguishable from plenty of other samples. Besides, we design an effective negative sample selection scheme to contain various intra and inter classes in the instance classification branch. Furthermore, we implement two variants of the proposed IAT, including a video-level one and an object-level one. They realize the concept of \textbf{instance} in different granularity as videos and target bounding boxes, respectively. The former enhances the ability to recognize the target from the background while the latter boosts the discriminative power for mitigating the target-distractor dilemma. Extensive experimental evaluations on 8 benchmark datasets show that both two versions of the proposed IAT achieve leading results against state-of-the-art methods while running at 30FPS. Code will be available when it is published.
BibTeX
@misc{wang2021explicitly,
  author = {Mengmeng Wang and Xiaoqian Yang and Yong Liu},
  title = {{Explicitly Modeling the Discriminability for Instance-Aware Visual Object Tracking}},
  year = {2021},
  eprint = {2110.15030},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.15030},
}
Figure