Target Focused Shallow Transformer Framework for Efficient Visual Tracking

Md. Maklachur RahmanAAAI 2024 · DOI

Abstract

Visual Object Tracking (VOT) requires a delicate balance between discriminative accuracy and computational efficiency. Traditional Siamese trackers offer high speed but often struggle with background clutter, while deep Transformer-based trackers provide high accuracy at the cost of significant computational overhead. This paper introduces a Target Focused Shallow Transformer framework that bridges this gap. By incorporating a Gaussian-inspired spatial mask into the self-attention mechanism and streamlining the transformer architecture, the proposed model effectively filters background noise and achieves state-of-the-art performance at real-time speeds (~34 FPS).

Introduction

The primary challenge in Visual Object Tracking is estimating a target's position across a video sequence despite distractions like background clutter, fast motion, or occlusions. Current methodologies generally fall into two categories: Siamese-based trackers, which are efficient but less discriminative, and Transformer-based trackers, which excel at modeling long-range dependencies but suffer from "feature mixing" where background noise degrades the target representation. This research seeks to create a tracker with the discriminative focus of a transformer and the efficiency required for practical real-time deployment.

Methodology

The proposed framework is built upon three core components:

  1. Siamese Backbone: A dual-stream convolutional network extracts features from the initial target template and the current search frame.
  2. Target-Focused Shallow Transformer: To mitigate background distraction, the self-attention mechanism is modified with a Gaussian map-inspired 2D binary mask. This mask forces the model to focus on the target within feature maps, effectively filtering out irrelevant information. The transformer is kept "shallow" to reduce parameter overhead and maximize processing speed.
  3. Refined Prediction Head: The architecture utilizes three branches—classification, regression, and center-ness. The center-ness branch specifically addresses a common failure in transformer trackers: inaccurate localization of the target's center, which typically leads to bounding box drift.

Experimental Results

Quantitative Performance

The framework was evaluated on major benchmarks including OTB100 and UAV123. As shown in the performance table below, the proposed model (labeled "Ours") consistently outperforms prominent trackers like TransT and SeqTrack in success and precision metrics.

Quantitative comparison of the proposed tracker against state-of-the-art methods on OTB100 and UAV123 benchmarks.

On the OTB100 dataset, the model achieved a 69.6% Success Score and a 90.3% Precision Score. On the UAV123 aerial benchmark, it reached a 70.2% Success Score. Crucially, the system operates at 34 Frames Per Second (FPS), ensuring real-time viability.

Qualitative Analysis

Qualitative evaluations demonstrate the robustness of the target-focused approach. In scenarios involving complex maneuvers—such as a motorcycle performing stunts or birds in flight—the proposed tracker maintains a tight lock on the target while competing models often drift toward background distractors.

Qualitative tracking results across challenging sequences involving fast motion, background clutter, and small targets.

Key Insights and Discussion

  • Spatial Discipline in Attention: Global self-attention can be a liability if it allows background noise to bleed into target features. The spatial mask restores the discipline needed to ignore clutter.
  • Efficiency via Shallowness: The results prove that a leaner, shallower transformer architecture can outperform deeper models if the attention mechanism is sufficiently focused on the target.
  • Localization Accuracy: The addition of a dedicated center-ness branch significantly reduces erroneous bounding box estimations, solving a persistent issue in transformer-based tracking.

Conclusion and Future Work

The Target Focused Shallow Transformer provides a robust solution for efficient visual tracking. Future research will explore adaptive attention mechanisms to dynamically adjust focus based on scene complexity and extend this "shallow transformer" philosophy to related tasks such as multi-object tracking and video object segmentation.

Summary updated

BibTeX
@inproceedings{rahman2024target,
  author = {Md. Maklachur Rahman},
  title = {{Target Focused Shallow Transformer Framework for Efficient Visual Tracking}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30405},
}
Figure