Release the Power of Online-Training for Robust Visual Tracking

Yifan Yang, Guorong Li, Yuankai Qi, Qingming Huang

Abstract

Modern visual tracking often relies on a "pre-train offline, finetune online" paradigm. However, online training is frequently hampered by data redundancy and feature drift, leading to overfitting and tracking failure. This paper presents a two-pronged strategy to enhance online training: a "sample squeezing" method using neighbor proximity to eliminate redundant data, and statistic-based losses to maximize feature discriminability. Experimental results on OTB and UAVDT benchmarks demonstrate significant improvements in robustness against occlusion and fast motion compared to state-of-the-art trackers.

Introduction

Visual object tracking is a fundamental task in computer vision, yet it remains challenging due to factors like target deformation, occlusion, and background clutter. Most contemporary trackers, such as MDNet, utilize a deep learning framework that is pre-trained on large datasets and then fine-tuned online to adapt to a specific target in a video sequence.

Despite its success, online fine-tuning faces two critical issues. First, data redundancy: the training sets generated during tracking are often dominated by "easy" samples that are nearly identical, causing the model to overfit to a specific appearance. Second, feature drift: as the tracker updates, the high-level features of the target and background may overlap, causing the classifier to lose its discriminative power. This paper addresses these challenges by optimizing both the training data and the loss functions used during online updates.

Methodology

The proposed framework enhances the standard MDNet-style tracking by introducing two core innovations: Sample Squeezing and Statistic-based Losses.

Overview of the proposed tracking framework, illustrating the transition from image space to semantic feature space where sample squeezing and statistic-based losses are applied.

Sample Squeezing with Neighbor Proximity

To combat data redundancy, the authors propose a "sample squeezing" technique. Traditional methods often use simple Euclidean distance to filter duplicates, which fails to account for the global distribution of the data. Instead, this method employs Neighbor Proximity.

For each sample, a "transmit-vector" is calculated, representing its distances to all other samples in the training set. By comparing these vectors, the algorithm identifies "global redundant samples"—those that do not contribute new information to the overall distribution. This allows the tracker to maintain a compact and diverse training set of approximately 150 high-quality samples, a significant reduction from the typical 3,000, without sacrificing performance.

Statistic-based Losses

To maintain a well-organized feature space and prevent drift, two auxiliary terms are added to the standard cross-entropy loss:

  1. Inter-class Distance Loss ($L_d$): This term acts as a "push" force, penalizing the model if the centers of the target and background clusters are too close in the feature space.
  2. Intra-class Variance Loss ($L_v$): This term acts as a "pull" force, encouraging samples of the same class to cluster tightly around their respective centers.

Together, these losses ensure that the "gap" between the target and background remains wide and distinct throughout the sequence, enhancing the classifier's robustness.

Experimental Evaluation

The proposed method was evaluated on several standard benchmarks, including OTB2013, OTB2015, and the UAVDT dataset.

Performance on OTB Benchmarks

On the OTB2015 dataset, the method outperformed the baseline MDNet and other competitive trackers like ECO and VITAL. The inclusion of statistic-based losses was shown to be crucial; ablation studies on OTB2013 revealed that the Intra-class Variance loss improved the overlap score (AUC) by 0.6%, while the Inter-class Distance loss added another 1.1%.

Qualitative tracking results on challenging sequences from the OTB benchmark. The proposed method (red) shows superior robustness compared to MDNet (green) and the ground truth (blue).

UAV Tracking

The tracker demonstrated exceptional performance on the UAVDT dataset. UAV tracking is particularly difficult due to small target sizes and cluttered backgrounds. The improved discriminative power provided by the statistic-based losses effectively prevented the tracker from drifting onto background clutter, a common failure mode for other trackers in this domain.

Discussion and Conclusion

The key insight of this work is that quality of training data often outweighs quantity. By reducing the training set size through intelligent squeezing, the learning signal becomes clearer and less diluted by redundant "easy" samples. Furthermore, "feature space engineering" via auxiliary losses ensures that the underlying data structure remains easily separable.

While the method achieves state-of-the-art robustness, it currently operates at approximately 1.0 FPS, which is not suitable for real-time applications. Future work may involve optimizing the architecture for speed or integrating these sample management strategies into more efficient frameworks, such as Correlation Filter-based trackers. Additionally, exploring automated ways to tune the hyperparameters for the loss terms could further enhance the system's versatility across different tracking scenarios.

Summary updated

Abstract

Convolutional neural networks (CNNs) have been widely adopted in the visual tracking community, significantly improving the state-of-the-art. However, most of them ignore the important cues lying in the distribution of training data and high-level features that are tightly coupled with the target/background classification. In this paper, we propose to improve the tracking accuracy via online training. On the one hand, we squeeze redundant training data by analyzing the dataset distribution in low-level feature space. On the other hand, we design statistic-based losses to increase the inter-class distance while decreasing the intra-class variance of high-level semantic features. We demonstrate the effectiveness on top of two high-performance tracking methods: MDNet and DAT. Experimental results on the challenging large-scale OTB2015 and UAVDT demonstrate the outstanding performance of our tracking method.
BibTeX
@inproceedings{yang2020release,
  author = {Yifan Yang and Guorong Li and Yuankai Qi and Qingming Huang},
  title = {{Release the Power of Online-Training for Robust Visual Tracking}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6956},
}
Figure