Learning motion blur robust vision transformers for real-time UAV tracking

You Wu, Xucheng Wang, Dan Zeng, Hengzhou Ye, Xiaolan Xie, Qijun Zhao, Shuiwang Li

TL;DR

This paper introduces BDTrack, a novel framework for real-time Unmanned Aerial Vehicle (UAV) tracking that significantly improves robustness to motion blur and enhances computational efficiency. By incorporating a dynamic early exiting mechanism and a motion blur robust learning strategy into Vision Transformers (ViTs), BDTrack achieves state-of-the-art performance with remarkable speed.

What Problem Does This Paper Address?

Tracking objects from Unmanned Aerial Vehicles (UAVs) presents unique challenges. The high speeds involved lead to significant motion blur, which degrades the performance of existing tracking algorithms. Furthermore, UAVs operate in resource-constrained environments, demanding efficient tracking solutions that minimize computational load and battery consumption. While Vision Transformers (ViTs) have shown promise in visual tracking, their inherent computational cost and lack of specific optimizations for UAV tracking limit their practical application. This paper aims to bridge this gap by developing a ViT-based tracker that is both efficient and robust to motion blur, making it suitable for real-time UAV applications.

Methods

The proposed BDTrack framework is built upon a single-stream ViT architecture and introduces two key innovations:

  • Motion Blur Robust Vision Transformers (MBRV): This component enhances the tracker's resilience to motion blur. It achieves this by minimizing the Mean Squared Error (MSE) between the feature representations of a clear target template and its synthetically blurred counterpart. This encourages the ViT to learn feature representations that are invariant to motion blur, ensuring consistent performance even under challenging visual conditions.
  • Dynamic Early Exit Module (DEEM): To improve computational efficiency, DEEM allows the ViT to dynamically exit its forward pass at earlier layers for less complex inputs. It assesses input complexity at each Transformer block and makes a decision to continue processing or exit based on a cumulative exiting score. This adaptive computation strategy conserves resources for simpler tasks, reserving full processing power for more challenging ones, without significantly impacting tracking accuracy.

Datasets

The paper evaluates BDTrack on four well-known UAV tracking benchmarks:

  • UAVDT [58]
  • VisDrone2018 [59]
  • UAV123 [60]
  • UAV123@10fps [60]

Key Insights and Ideas

  • Motion blur is a critical bottleneck for UAV tracking: The paper convincingly demonstrates how motion blur severely degrades tracking performance, highlighting the need for specialized solutions.
  • ViTs can be made motion blur robust efficiently: The MBRV component shows that learning feature invariance through a simple MSE loss is effective and doesn't add significant computational overhead during inference.
  • Dynamic early exiting is key for real-time ViT tracking: DEEM effectively balances computational cost and performance by adaptively allocating resources, making ViTs practical for resource-constrained platforms like UAVs.
  • A unified framework is powerful: Combining MBRV and DEEM into a single BDTrack framework yields synergistic benefits, achieving both robustness and efficiency.

Key Results

  • State-of-the-art performance: BDTrack-DeiT achieves an average precision of 0.844 and runs at approximately 283 FPS on four UAV tracking benchmarks, setting a new record.
  • Motion blur robustness: BDTrack-DeiT significantly outperforms other trackers on motion blur and camera motion subsets of the UAVDT dataset, showing improvements of up to 7.0%/5.0% in precision/success rate compared to a baseline without the proposed components.
  • Efficiency: BDTrack-DeiT achieves the highest GPU speed (283.4 FPS) among tested trackers, outperforming competitors like Aba-ViTrack by 1.6 times on GPU and 1.3 times on CPU. It also significantly outperforms deep trackers like ROMTrack and EVPTrack by factors of 5 and 11, respectively, in speed.
  • Qualitative results: Visualizations show that BDTrack-DeiT successfully tracks targets in challenging scenarios involving camera motion, occlusion, and background clutter, maintaining a center location error below 20 pixels.

Limitations and Future Work

  • Limitations:
    • The current design lacks an explicit mechanism for recovering target features after full occlusion, which can lead to identity loss.
  • Performance degrades when dealing with extremely small or visually similar objects, attributed to limited spatial resolution of high-level features and the absence of fine-grained contrast enhancement.
  • Future Work:
    • Enhance robustness in scenarios with full occlusion and low visibility by incorporating lightweight memory mechanisms for re-identifying targets after temporary disappearance.
  • Improve model sensitivity to fine-grained visual cues by integrating high-resolution refinement modules for more accurate localization of small targets.

Summary updated

Abstract

Unmanned aerial vehicle (UAV) tracking is critical for applications like surveillance, search-and-rescue, and autonomous navigation. However, the high-speed movement of UAVs and targets introduces unique challenges, including real-time processing demands and severe motion blur, which degrade the performance of existing generic trackers. While single-stream vision transformer (ViT) architectures have shown promise in visual tracking, their computational inefficiency and lack of UAV-specific optimizations limit their practicality in this domain. In this paper, we boost the efficiency of this framework by tailoring it into an adaptive computation framework that dynamically exits Transformer blocks for real-time UAV tracking. The motivation behind this is that tracking tasks with fewer challenges can be adequately addressed using low-level feature representations. Simpler tasks can often be handled with less demanding, lower-level features. This approach allows the model use computational resources more efficiently by focusing on complex tasks and conserving resources for easier ones. Another significant enhancement introduced in this paper is the improved effectiveness of ViTs in handling motion blur, a common issue in UAV tracking caused by the fast movements of either the UAV, the tracked objects, or both. This is achieved by acquiring motion blur robust representations through enforcing invariance in the feature representation of the target with respect to simulated motion blur. We refer to our proposed approach as BDTrack. Extensive experiments conducted on four tracking benchmarks validate the effectiveness and versatility of our approach, demonstrating its potential as a practical and effective approach for real-time UAV tracking. Code is released at: https://github.com/wuyou3474/BDTrack.
BibTeX
@article{wu2026learning,
  author = {You Wu and Xucheng Wang and Dan Zeng and Hengzhou Ye and Xiaolan Xie and Qijun Zhao and Shuiwang Li},
  title = {{Learning motion blur robust vision transformers for real-time UAV tracking}},
  journal = {Expert Systems with Applications},
  year = {2026},
  doi = {10.1016/j.eswa.2025.129445},
}
Figure