Abstract
Research in Anti-UAV (Unmanned Aerial Vehicle) tracking has explored various modalities, including RGB, TIR, and RGB-T fusion. However, a unified framework for cross-modal collaboration is still lacking. Existing approaches have primarily focused on independent models for individual tasks, often overlooking the potential for cross-modal information sharing. Furthermore, Anti-UAV tracking techniques are still in their infancy, with current solutions struggling to achieve effective multimodal data fusion. To address these challenges, we propose UAUTrack, a unified single-target tracking framework built upon a single-stream, single-stage, end-to-end architecture that effectively integrates multiple modalities. UAUTrack introduces a key component: a text prior prompt strategy that directs the model to focus on UAVs across various scenarios. Experimental results show that UAUTrack achieves state-of-the-art performance on the Anti-UAV and DUT Anti-UAV datasets, and maintains a favourable trade-off between accuracy and speed on the Anti-UAV410 dataset, demonstrating both high accuracy and practical efficiency across diverse Anti-UAV scenarios.
Introduction
Tracking drones from the ground is a significant challenge due to their small size, rapid movement, and the need to operate in cluttered environments or low-visibility conditions. While standard RGB cameras and Thermal Infrared (TIR) sensors are commonly used, existing solutions are often fragmented. Most current methods utilize independent models for different sensors or rely on complex, multi-stage "hybrid" pipelines that combine separate detection and tracking modules. This fragmentation prevents effective information sharing across modalities and creates significant computational overhead, hindering real-time deployment on edge devices.
Methods
UAUTrack departs from traditional Siamese or detection-based architectures by employing a Unified Single-Stream Architecture. Instead of processing template and search images in separate branches, it uses a single-stream transformer backbone (based on Fast-iTPN). This allows the model to perform feature extraction and fusion simultaneously, fostering better interaction between the target template and the search area.
The framework is modality-adaptive, handling RGB, TIR, and fused RGB-T data by preprocessing them into a consistent format. For multimodal RGB-T, images are concatenated into a 6-channel input, allowing the transformer to learn cross-modal dependencies directly.
A core innovation is the Text Prior Prompt (TPP) strategy. Using a frozen CLIP-L language model, UAUTrack generates embeddings from text prompts such as "track a tiny/small/medium/normal drone." These linguistic features are injected into the transformer's encoder layers to provide "semantic guidance," helping the visual model focus on the specific scale and characteristics of the target drone without requiring an external object detector.
Key Results
UAUTrack demonstrates state-of-the-art performance across several major benchmarks. On the Anti-UAV (TIR) dataset, it achieved an AUC (Success) of 68.8%, surpassing previous methods like FocusTrack. On the Anti-UAV410 benchmark, it achieved a State Accuracy (SA) of 66.3% while running at 45 FPS—nearly six times faster than competing global trackers like SiamDT.
Ablation studies highlight the value of semantic guidance: the inclusion of the TPP module provided a ~5% boost in AUC compared to the visual-only baseline. This suggests that language acts as a powerful spatial prior, helping the attention mechanism "zoom in" on the correct feature scales even in challenging "thermal crossover" scenarios where a drone's heat signature matches the background.
Conclusion
By eliminating the need for separate re-detection modules and unifying the tracking pipeline, UAUTrack achieves a superior balance of speed and accuracy. Future work will explore moving beyond fixed templates to more descriptive, dynamic text prompts that account for drone color, type, or specific maneuvers, as well as further optimizing the architecture for low-power edge hardware.
Summary updated