Abstract
Visual object tracking requires continuous model updates to adapt to target appearance changes. However, limited memory often forces trackers to rely on naive sample selection strategies, which are prone to redundancy and noise from occlusions or blur. This paper introduces UATracker, a novel framework that leverages model uncertainty to intelligently select significant training samples. By filtering unreliable data and employing mixed-feature data augmentation, UATracker achieves state-of-the-art performance on major benchmarks, including OTB100 and VOT2018, while maintaining real-time speeds of 45 FPS.
Introduction
Visual object tracking involves following a specific target across a video sequence. Modern trackers utilize "online learning" to update their internal models as the video progresses. Despite its benefits, online learning faces a critical bottleneck: memory constraints. Trackers can typically only store a small subset of frames for training.
Existing methods often employ a naive selection strategy, prioritizing the initial frame and the most recent ones. This approach leads to two primary issues: redundancy, where temporally adjacent frames provide little new information, and noise, where the inclusion of blurred or occluded frames causes "tracking drift." UATracker addresses these challenges by using the model's own uncertainty to guide sample selection.
Methods
The UATracker architecture integrates uncertainty estimation directly into the tracking pipeline to improve the quality of the online training set.
Significant Sample Selection (S3)
The core innovation is the Significant Sample Selection (S3) module. The tracker's regression branch predicts both the target's location and an estimate of aleatoric uncertainty (inherent data noise). The video sequence is divided into temporal clusters, and S3 calculates an "uncertainty-adjusted center" for each.
As shown in the figure above, S3 filters out "garbage" samples—such as those with high uncertainty due to occlusion—and selects the frame closest to the cluster center with a sufficiently high confidence score. This ensures the training set remains diverse and representative of the target's true appearance over time.
Mixed-feature Data Augmentation and DCN
To further enhance robustness, the authors propose Mixed-feature Data Augmentation. Unlike standard pixel-level augmentation, this technique blends feature maps from different frames during offline training to simulate complex scenarios like partial occlusions. Additionally, the backbone (ResNet50) is enhanced with Deformable Convolutions (DCN), allowing the model to adapt its receptive field to objects undergoing rotation or scale changes.
Key Results and Discussion
UATracker was evaluated on major benchmarks including LaSOT, UAV123, OTB100, and VOT2018. It consistently outperformed the previous state-of-the-art, DiMP50:
- OTB100: Achieved an AUC of 70.9%, a 2.5% improvement over DiMP50. On sequences specifically tagged with "occlusion," the performance gain rose to +5.2%, validating the effectiveness of the S3 strategy.
- VOT2018: Reached a leading Expected Average Overlap (EAO) score of 0.458.
- Efficiency: The tracker operates at 45 FPS on a standard GTX 1080ti, meeting the requirements for real-time application.
Conclusion
The primary insight of this work is that a model’s own "doubt" can serve as a powerful quality control mechanism. By using uncertainty as a filter, UATracker maintains a high-quality temporal memory of the target. Future work may involve more sophisticated, content-aware clustering logic to further reduce redundancy and dynamic thresholding to adapt to varying video difficulties.
Summary updated