Learning Motion Blur Robust Vision Transformers with Dynamic Early Exit for Real-Time UAV Tracking

You Wu, Xucheng Wang, Dan Zeng, Hengzhou Ye, Xiaolan Xie, Qijun Zhao, Shuiwang Li

Abstract

Recently, the surge in the adoption of single-stream architectures utilizing pre-trained ViT backbones represents a promising advancement in the field of generic visual tracking. By integrating feature extraction and fusion into a cohesive framework, these architectures offer improved performance, efficiency, and robustness. However, there has been limited exploration into optimizing these frameworks for UAV tracking. In this paper, we boost the efficiency of this framework by tailoring it into an adaptive computation framework that dynamically exits Transformer blocks for real-time UAV tracking. The motivation behind this is that tracking tasks with fewer challenges can be adequately addressed using low-level feature representations. This approach allows the model to use computational resources more efficiently by focusing on complex tasks and conserving resources for easier ones. Another significant enhancement introduced in this paper is the improved effectiveness of ViTs in handling motion blur, a common issue in UAV tracking caused by the fast movements of either the UAV, the tracked objects, or both. This is achieved by acquiring motion blur robust representations through enforcing invariance in the feature representation of the target with respect to simulated motion blur. The proposed approach is dubbed BDTrack. Extensive experiments conducted on five tracking benchmarks validate the effectiveness and versatility of our approach, establishing it as a cutting-edge solution in real-time UAV tracking.

Introduction

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in critical applications such as search-and-rescue, surveillance, and autonomous navigation. However, visual tracking from a drone platform faces two primary hurdles: Motion Blur and Resource Constraints. High-speed maneuvers by either the drone or the target create significant blur, causing standard trackers to lose their target lock. Furthermore, drones have limited battery life and onboard processing power, making computationally heavy Vision Transformers (ViTs) difficult to deploy in real-time. Existing solutions often sacrifice speed for accuracy or introduce complex deblurring modules that add prohibitive latency.

The BDTrack Framework

To address these challenges, the researchers developed BDTrack, a framework that integrates two primary innovations into a single-stream ViT backbone to achieve both robustness and efficiency.

Learning Motion Blur Robust Representations (MBRV)

Rather than attempting to "fix" a blurry image through slow reconstruction modules, BDTrack trains the model to extract consistent features regardless of image clarity. During training, motion blur is simulated on the target template. A Mean Squared Error (MSE) loss is then applied to force the feature representations of sharp and blurred versions to be nearly identical. This "feature invariance" approach ensures the tracker remains robust to blur during flight with zero additional computational overhead.

Dynamic Early Exit Module (DEEM)

Recognizing that not every video frame requires the full depth of a Transformer, the authors introduced the Dynamic Early Exit Module (DEEM). This module places "exit gates" after Transformer blocks. If the model achieves sufficient confidence in its prediction after a few layers, it exits early, skipping unnecessary computation. This structured efficiency allows the model to focus resources on difficult frames (e.g., occlusions) while maintaining high speed on simpler ones.

The BDTrack architecture featuring the Dynamic Early Exit Module (DEEM) and Motion Blur Robust Representations (MBRV) learning.

Experimental Results

BDTrack was evaluated across major UAV tracking benchmarks, including UAV123, VisDrone2018, and UAVDT. The results demonstrate a superior balance of precision and real-time performance.

  • Blazing Speed: BDTrack-DeiT achieves an average of 283.4 FPS on a TitanX GPU, outperforming previous efficient trackers like Aba-ViTrack by approximately 1.6x.
  • State-of-the-Art Precision: The model set a new benchmark with an average precision of 0.844 across the tested datasets.
  • Robustness to Blur: On the motion blur subset of the UAVDT dataset, BDTrack outperformed specialized deblurring trackers (like ABDNet) by 5.5% in precision while operating at twice the speed.
  • Embedded Deployment: On an NVIDIA Jetson AGX Xavier, the model maintained a usable 44 FPS, proving its readiness for real-world drone hardware.
Comparison of precision and speed (FPS) against state-of-the-art trackers on UAV benchmarks.

Conclusion

BDTrack represents a significant step forward for UAV-based visual tracking. By prioritizing feature invariance over image reconstruction and utilizing structured early exiting, the framework achieves record-breaking speeds without compromising precision. Future work will focus on integrating lightweight memory mechanisms to handle long-term occlusions and high-resolution refinement modules for tracking miniature targets.

Summary updated

Abstract

Recently, the surge in the adoption of single-stream architectures utilizing pre-trained ViT backbones represents a promising advancement in the field of generic visual tracking. By integrating feature extraction and fusion into a cohesive framework, these architectures offer improved performance, efficiency, and robustness. However, there has been limited exploration into optimizing these frameworks for UAV tracking. In this paper, we boost the efficiency of this framework by tailoring it into an adaptive computation framework that dynamically exits Transformer blocks for real-time UAV tracking. The motivation behind this is that tracking tasks with fewer challenges can be adequately addressed using low-level feature representations. Simpler tasks can often be handled with less demanding, lower-level features. This approach allows the model use computational resources more efficiently by focusing on complex tasks and conserving resources for easier ones. Another significant enhancement introduced in this paper is the improved effectiveness of ViTs in handling motion blur, a common issue in UAV tracking caused by the fast movements of either the UAV, the tracked objects, or both. This is achieved by acquiring motion blur robust representations through enforcing invariance in the feature representation of the target with respect to simulated motion blur. The proposed approach is dubbed BDTrack. Extensive experiments conducted on five tracking benchmarks validate the effectiveness and versatility of our approach, establishing it as a cutting-edge solution in real-time UAV tracking. Code is released at: https://github.com/wuyou3474/BDTrack.
BibTeX
@article{wu2026learning,
  author = {You Wu and Xucheng Wang and Dan Zeng and Hengzhou Ye and Xiaolan Xie and Qijun Zhao and Shuiwang Li},
  title = {{Learning Motion Blur Robust Vision Transformers with Dynamic Early Exit for Real-Time UAV Tracking}},
  journal = {Expert Systems with Applications},
  year = {2026},
  doi = {10.1016/j.eswa.2025.129445},
}
Figure