Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Tri-Attentional Correlation Filters

Yujie He, Changhong Fu, Fuling Lin, Yiming Li, Peng Lu

Abstract

Object tracking has been broadly applied in unmanned aerial vehicle (UAV) tasks in recent years. However, existing algorithms still face difficulties such as partial occlusion, clutter background, and other challenging visual factors. Inspired by the cutting-edge attention mechanisms, a novel object tracking framework is proposed to leverage multi-level visual attention. Three primary attention, i.e., contextual attention, dimensional attention, and spatiotemporal attention, are integrated into the training and detection stages of correlation filter-based tracking pipeline. Therefore, the proposed tracker is equipped with robust discriminative power against challenging factors while maintaining high operational efficiency in UAV scenarios. Quantitative and qualitative experiments on two well-known benchmarks with 173 challenging UAV video sequences demonstrate the effectiveness of the proposed framework. The proposed tracking algorithm favorably outperforms 12 state-of-the-art methods, yielding 4.8% relative gain in UAVDT and 8.2% relative gain in UAV123@10fps against the baseline tracker while operating at the speed of ~28 frames per second.

Introduction

Unmanned Aerial Vehicles (UAVs) rely heavily on visual object tracking for critical tasks such as autonomous landing, target following, and surveillance. Despite its importance, aerial tracking remains a significant challenge due to sudden viewpoint changes, partial occlusions, and cluttered backgrounds. For instance, tracking a specific vehicle through a busy intersection from a high-altitude drone requires distinguishing the target from numerous similar-looking distractors.

Furthermore, drones are constrained by limited battery life and onboard computing power. While deep learning models offer high accuracy, their computational demands often make them impractical for real-time deployment on standard UAV hardware. Correlation Filter (CF)-based trackers have emerged as a popular alternative due to their efficiency in the Fourier domain. However, standard CF trackers are prone to "drifting"—a phenomenon where the filter erroneously learns background noise as part of the target during occlusions or in cluttered scenes, eventually leading to tracking failure.

The Tri-Attentional Correlation Filter (TACF) Framework

To address these challenges, we propose the Tri-Attentional Correlation Filter (TACF) framework. This approach builds upon the Kernel Cross-Correlator (KCC) baseline and introduces three distinct attention mechanisms to enhance robustness during both the training and detection stages.

Overview of the proposed TACF framework featuring contextual, dimensional, and spatiotemporal attention modules.

1. Contextual Attention (Training Stage)

Standard trackers often focus solely on the target's appearance. TACF incorporates contextual attention by extracting patches from the surrounding background (up, down, left, and right) to learn what not to track. To prevent the model from learning corrupted data during occlusions, we introduce the Peak Median Energy ratio (PME). This metric evaluates the sharpness of the tracking response map. A drop in PME indicates potential confusion or occlusion, prompting the system to dynamically lower the penalty factor for background patches and prevent noisy model updates.

2. Dimensional Attention (Detection Stage)

The tracker utilizes multi-dimensional features, specifically Histogram of Oriented Gradients (HOG) and Color Names. Since the relevance of these features varies frame-by-frame, the dimensional attention module calculates weights for each channel based on their response characteristics. This allows the tracker to actively filter out noisy or irrelevant feature channels, focusing on the most discriminative cues for the current scene.

3. Spatiotemporal Attention (Detection Stage)

To maintain physical consistency, the spatiotemporal attention module leverages the object's previous motion (velocity and shift) to predict its likely current location. By applying a spatial mask (a Hanning window) to the response map, the system suppresses high-confidence "peaks" that are spatially inconsistent with the target's predicted trajectory, thereby mitigating the risk of jumping to a similar-looking distractor.

Experimental Evaluation

The TACF tracker was rigorously evaluated on two major UAV tracking benchmarks: UAV123@10fps (123 sequences, >37,000 frames) and UAVDT (50 complex sequences). These datasets include challenging attributes such as scale variation, fast motion, and illumination changes.

Success plots on the UAVDT and UAV123@10fps benchmarks comparing TACF against state-of-the-art trackers.

As shown in the success plots, TACF consistently outperforms 12 state-of-the-art trackers, including ECO_HC, STRCF, and SRDCF. On the UAVDT benchmark, TACF achieved an Area Under Curve (AUC) success rate of 0.437, representing an 8.4% improvement over the baseline KCC. On UAV123@10fps, it achieved an AUC of 0.463, a 21.1% gain over the baseline.

Crucially, TACF maintains high operational efficiency. It operates at an average of 28.1 Frames Per Second (FPS) on a standard CPU (Intel i7-8700K). This real-time performance, combined with its robustness against background clutter and occlusion, makes it highly suitable for practical UAV applications.

Conclusion

The TACF framework demonstrates that multi-level attention mechanisms can significantly boost the performance of traditional correlation filter trackers without requiring the heavy overhead of deep neural networks. By intelligently managing contextual, dimensional, and spatiotemporal information, TACF achieves state-of-the-art results on challenging UAV benchmarks.

Future work will focus on optimizing the implementation for onboard deployment using C++ and exploring the integration of lightweight, mobile-friendly CNN features to further enhance semantic understanding while preserving real-time speeds.

Summary updated

Abstract

Object tracking has been broadly applied in unmanned aerial vehicle (UAV) tasks in recent years. However, existing algorithms still face difficulties such as partial occlusion, clutter background, and other challenging visual factors. Inspired by the cutting-edge attention mechanisms, a novel object tracking framework is proposed to leverage multi-level visual attention. Three primary attention, i.e., contextual attention, dimensional attention, and spatiotemporal attention, are integrated into the training and detection stages of correlation filter-based tracking pipeline. Therefore, the proposed tracker is equipped with robust discriminative power against challenging factors while maintaining high operational efficiency in UAV scenarios. Quantitative and qualitative experiments on two well-known benchmarks with 173 challenging UAV video sequences demonstrate the effectiveness of the proposed framework. The proposed tracking algorithm favorably outperforms 12 state-of-the-art methods, yielding 4.8% relative gain in UAVDT and 8.2% relative gain in UAV123@10fps against the baseline tracker while operating at the speed of $\sim$ 28 frames per second.
BibTeX
@inproceedings{he2020towards,
  author = {Yujie He and Changhong Fu and Fuling Lin and Yiming Li and Peng Lu},
  title = {{Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Tri-Attentional Correlation Filters}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  doi = {10.1109/iros45743.2020.9341784},
}
Figure