Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Tri-Attentional Correlation Filters

Yujie He, Changhong Fu, Fuling Lin, Yiming Li, Peng Lu

TL;DR

Tracking objects from drones is notoriously difficult due to background clutter, occlusions, and strict limits on onboard computing power. This paper introduces a Tri-Attentional Correlation Filter (TACF) framework that uses contextual, dimensional, and spatiotemporal attention mechanisms to boost tracking robustness. By intelligently focusing on the most reliable visual cues, TACF achieves state-of-the-art accuracy on major UAV benchmarks while running in real-time (~28 FPS) on a standard CPU.

What Problem Does This Paper Address?

Unmanned Aerial Vehicles (UAVs) rely heavily on visual object tracking for tasks like autonomous landing, target following, and surveillance. However, aerial tracking is plagued by sudden viewpoint changes, partial occlusions, and cluttered backgrounds (e.g., tracking a car driving through a busy intersection).

Because drones have limited battery and computing power, heavy Deep Learning models are often impractical. Correlation Filter (CF)-based trackers are a popular alternative because they use highly efficient Fourier domain mathematics to learn object appearances. Unfortunately, standard CF trackers are prone to "drifting." If the background looks similar to the target, or if the target is temporarily occluded, the filter accidentally learns the background noise as part of the object, eventually losing the target entirely. Existing attempts to make CF trackers "background-aware" often sample too much useless data, wasting precious computational resources.

Methods

The authors build upon a baseline CF tracker called the Kernel Cross-Correlator (KCC) and introduce a "Tri-Attention" framework. This framework applies attention mechanisms at both the filter training stage and the response generation stage:

  1. Contextual Attention (Training Stage): Instead of just learning what the object looks like, the tracker also extracts patches from the surrounding background (up, down, left, right) to learn what not to track. To prevent the filter from learning bad data during an occlusion, the authors introduce a novel metric called the Peak Median Energy ratio (PME). PME evaluates the sharpness of the tracking response map. If the PME drops (indicating the tracker is confused), the system dynamically lowers the penalty factor for background patches, preventing the filter from being corrupted by noisy updates.
  2. Dimensional Attention (Detection Stage): The tracker uses multi-dimensional features (HOG and Color Names). Not all feature channels are equally useful in every frame. This module calculates a weight for each dimension based on its maximum and average response, actively filtering out noisy or irrelevant feature channels.
  3. Spatiotemporal Attention (Detection Stage): This module uses the object's previous motion (velocity and shift) to predict its current location. It applies a spatial mask (a Hanning window) to the response map, suppressing high-confidence "peaks" that appear too far away from the predicted location to be physically possible.

Datasets

The proposed TACF tracker was evaluated on two established, highly challenging UAV tracking benchmarks:

  • UAV123@10fps: Contains 123 video sequences with over 37,000 frames, featuring 12 kinds of challenging visual attributes (like scale variation and fast motion).
  • UAVDT: Contains 50 complex scenario sequences fully annotated with bounding boxes and 9 kinds of visual attributes.

Key Insights and Ideas

  • Dynamic health-checks prevent drift: The introduction of the PME metric is a standout idea. By mathematically quantifying the "confidence" of the response map, the tracker knows when to trust its eyes and when to stop updating its model. This is crucial for surviving temporary occlusions.
  • Attention cleans up traditional features: Deep learning isn't the only way to achieve high accuracy. By applying smart, multi-level attention mechanisms to clean up the signal, traditional hand-crafted features (like HOG and Color Names) can still achieve state-of-the-art results without the massive compute overhead of a CNN.
  • Synergy between training and detection: Contextual attention improves the discriminative power of the filter itself, while dimensional and spatiotemporal attention refine the output (the response map). This two-pronged approach effectively isolates the target from the noise.

Key Results

TACF was compared against 12 state-of-the-art trackers (including ECO_HC, STRCF, and SRDCF).

  • UAVDT Benchmark: TACF ranked #1 with an Area Under Curve (AUC) success rate of 0.437, outperforming the baseline KCC tracker by 8.4% and beating the second-best tracker (SRDCF) by 4.3%.
  • UAV123@10fps Benchmark: TACF ranked #1 with an AUC of 0.463, outperforming the baseline KCC by a massive 21.1%.
  • Speed: Despite the added attention modules, TACF operates at an average of 28.1 Frames Per Second (FPS) on a single CPU (Intel i7-8700K), making it highly viable for real-time UAV applications.
  • Qualitative: Ablation studies showed TACF excels specifically in scenarios with partial occlusion and background clutter, where competing trackers frequently lose the target.

Limitations and Future Work

  • Implementation Overhead: The current version is written in MATLAB. For actual deployment on a drone's onboard processor, it needs to be optimized and rewritten in C++ with parallel computing techniques.
  • Large Object Slowdown: The contextual attention module extracts multiple background patches. If the tracked object is very large, processing these surrounding patches slows down the tracking speed.
  • Feature Limitations: While hand-crafted features keep the tracker fast, they have a ceiling on semantic understanding. The authors suggest that future work could integrate lightweight, mobile-friendly Convolutional Neural Networks (CNNs) to extract deeper semantic features without sacrificing real-time performance.

Summary updated

Abstract

Object tracking has been broadly applied in unmanned aerial vehicle (UAV) tasks in recent years. However, existing algorithms still face difficulties such as partial occlusion, clutter background, and other challenging visual factors. Inspired by the cutting-edge attention mechanisms, a novel object tracking framework is proposed to leverage multi-level visual attention. Three primary attention, i.e., contextual attention, dimensional attention, and spatiotemporal attention, are integrated into the training and detection stages of correlation filter-based tracking pipeline. Therefore, the proposed tracker is equipped with robust discriminative power against challenging factors while maintaining high operational efficiency in UAV scenarios. Quantitative and qualitative experiments on two well-known benchmarks with 173 challenging UAV video sequences demonstrate the effectiveness of the proposed framework. The proposed tracking algorithm favorably outperforms 12 state-of-the-art methods, yielding 4.8% relative gain in UAVDT and 8.2% relative gain in UAV123@10fps against the baseline tracker while operating at the speed of $\sim$ 28 frames per second.
BibTeX
@inproceedings{he2020towards,
  author = {Yujie He and Changhong Fu and Fuling Lin and Yiming Li and Peng Lu},
  title = {{Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Tri-Attentional Correlation Filters}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  doi = {10.1109/iros45743.2020.9341784},
}

Loading related content...

Figure