Abstract
Visual object tracking often suffers from a trade-off between speed and accuracy, as well as the risk of "drift" when a tracker incorrectly learns background features as part of the target. We present the Uncertainty Sampling co-Tracker (UST), a framework that addresses these challenges through an asymmetric co-tracking approach. UST pairs a fast, adaptive $k$-nearest neighbor (KNN) classifier with a robust "oracle" detector. By utilizing active learning to query the oracle only for the most uncertain samples and implementing a budgeting mechanism to prune redundant data, UST achieves state-of-the-art performance on the OTB-100 benchmark (0.52 AUC) while maintaining real-time speeds of 28.3 fps.
Introduction
Modern visual tracking is frequently framed as a binary classification problem, distinguishing the target from the background. However, this approach faces three primary hurdles:
- Drift: Inaccurate updates cause the tracker to gradually lose the target.
- Computational Complexity: Accumulating training samples over time slows down the system.
- The Speed-Accuracy Gap: Robust detectors are typically too slow for real-time tracking, while fast detectors are prone to errors.
The UST framework bridges this gap by creating a "student-teacher" relationship between a fast detector and a sophisticated oracle.
The UST Framework
Asymmetric Co-Tracking
The system employs two distinct classifiers with different strengths. The Rapid Detector is a KNN classifier with short-term memory, designed for speed and quick adaptation to appearance changes. The Oracle is a more complex, part-based detector with long-term memory, providing robustness against occlusions and noise.
Uncertainty Sampling
Rather than having both trackers process every frame, UST uses active learning. The KNN identifies "uncertain" samples—those lying near its decision boundary where it cannot confidently distinguish target from background. It queries the Oracle only for these difficult patches. This "scaffolding" allows the fast learner to improve its decision boundaries without the constant overhead of the Oracle.
Budgeting and Memory Management
To maintain real-time performance, UST incorporates a budgeting mechanism with five accounting rules. These rules prune the sample pool by discarding redundant data, removing old samples via a timer, and identifying outliers. This "forgetting mechanism" is crucial for escaping local minima caused by distractors.
Experimental Results
The authors evaluated UST on the OTB-100 benchmark, covering 100 sequences with challenges like motion blur, illumination changes, and occlusions.
- Accuracy: UST achieved a success plot AUC of 0.52, outperforming STRUCK (0.48), TLD (0.46), and MIL (0.38).
- Speed: The system averaged 28.3 fps on a standard i7 processor, meeting real-time requirements.
- Robustness: UST showed superior performance in 9 out of 11 challenge categories, particularly excelling in handling rotations and deformations.
Conclusion
UST demonstrates that selective communication between classifiers is more effective than full data exchange. By focusing on the most informative samples and actively managing memory, the tracker achieves a balance of speed and robustness. Future work may involve incorporating context information and more advanced features to better handle extreme background clutter and low-resolution targets.
Summary updated