Abstract
Visual Object Tracking (VOT) requires a delicate balance between discriminative accuracy and computational efficiency. Traditional Siamese trackers offer high speed but often struggle with background clutter, while deep Transformer-based trackers provide high accuracy at the cost of significant computational overhead. This paper introduces a Target Focused Shallow Transformer framework that bridges this gap. By incorporating a Gaussian-inspired spatial mask into the self-attention mechanism and streamlining the transformer architecture, the proposed model effectively filters background noise and achieves state-of-the-art performance at real-time speeds (~34 FPS).
Introduction
The primary challenge in Visual Object Tracking is estimating a target's position across a video sequence despite distractions like background clutter, fast motion, or occlusions. Current methodologies generally fall into two categories: Siamese-based trackers, which are efficient but less discriminative, and Transformer-based trackers, which excel at modeling long-range dependencies but suffer from "feature mixing" where background noise degrades the target representation. This research seeks to create a tracker with the discriminative focus of a transformer and the efficiency required for practical real-time deployment.
Methodology
The proposed framework is built upon three core components:
- Siamese Backbone: A dual-stream convolutional network extracts features from the initial target template and the current search frame.
- Target-Focused Shallow Transformer: To mitigate background distraction, the self-attention mechanism is modified with a Gaussian map-inspired 2D binary mask. This mask forces the model to focus on the target within feature maps, effectively filtering out irrelevant information. The transformer is kept "shallow" to reduce parameter overhead and maximize processing speed.
- Refined Prediction Head: The architecture utilizes three branches—classification, regression, and center-ness. The center-ness branch specifically addresses a common failure in transformer trackers: inaccurate localization of the target's center, which typically leads to bounding box drift.
Experimental Results
Quantitative Performance
The framework was evaluated on major benchmarks including OTB100 and UAV123. As shown in the performance table below, the proposed model (labeled "Ours") consistently outperforms prominent trackers like TransT and SeqTrack in success and precision metrics.
On the OTB100 dataset, the model achieved a 69.6% Success Score and a 90.3% Precision Score. On the UAV123 aerial benchmark, it reached a 70.2% Success Score. Crucially, the system operates at 34 Frames Per Second (FPS), ensuring real-time viability.
Qualitative Analysis
Qualitative evaluations demonstrate the robustness of the target-focused approach. In scenarios involving complex maneuvers—such as a motorcycle performing stunts or birds in flight—the proposed tracker maintains a tight lock on the target while competing models often drift toward background distractors.
Key Insights and Discussion
- Spatial Discipline in Attention: Global self-attention can be a liability if it allows background noise to bleed into target features. The spatial mask restores the discipline needed to ignore clutter.
- Efficiency via Shallowness: The results prove that a leaner, shallower transformer architecture can outperform deeper models if the attention mechanism is sufficiently focused on the target.
- Localization Accuracy: The addition of a dedicated center-ness branch significantly reduces erroneous bounding box estimations, solving a persistent issue in transformer-based tracking.
Conclusion and Future Work
The Target Focused Shallow Transformer provides a robust solution for efficient visual tracking. Future research will explore adaptive attention mechanisms to dynamically adjust focus based on scene complexity and extend this "shallow transformer" philosophy to related tasks such as multi-object tracking and video object segmentation.
Summary updated