Abstract
Visual object tracking on Unmanned Aerial Vehicles (UAVs) requires a delicate balance between high accuracy and computational efficiency, as these platforms often rely on resource-constrained embedded CPUs. This paper introduces the Siamese Transformer Pyramid Network (SiamTPN), a novel architecture designed to bring the representational power of Transformers to real-time UAV tracking. By integrating a lightweight CNN backbone with a feature pyramid and a specialized "Pooling Attention" mechanism, SiamTPN captures global context without the quadratic complexity of standard Transformers. Experimental results demonstrate that SiamTPN achieves state-of-the-art performance on benchmarks like UAV123 and LaSOT, maintaining a frame rate of over 30 FPS on a single CPU core.
Introduction
The deployment of UAVs for surveillance, border security, and autonomous navigation necessitates robust visual tracking. While deep learning has significantly advanced tracking capabilities, most high-performing models are optimized for powerful GPUs. In contrast, UAVs typically operate on embedded systems where CPU resources are the primary bottleneck. Existing lightweight trackers often fail to handle complex environmental changes because their shallow architectures lack the global context necessary for discriminative feature extraction. SiamTPN addresses this gap by providing a "smart" yet "lean" solution capable of running efficiently on a drone's onboard processor.
Methodology
The SiamTPN architecture is built upon three core pillars: a Siamese backbone, a Transformer Pyramid Network (TPN), and a Pooling Attention (PA) module.
- Siamese Backbone: The model utilizes a lightweight ShuffleNetV2 backbone to extract multi-scale features from both the target template and the search region.
- Transformer Pyramid Network (TPN): Unlike traditional trackers that rely on a single feature layer, SiamTPN leverages a feature pyramid ($P_3, P_4, P_5$). The middle-level features ($P_4$) act as a query to aggregate information from low-level textures ($P_3$) and high-level semantic features ($P_5$), creating a more robust representation.
- Pooling Attention (PA) Module: To overcome the quadratic computational cost of standard Transformer attention, the PA module introduces a pooling layer for the "Keys" and "Values." This reduction in dimensionality allows the model to capture global relationships with significantly lower memory and time complexity, making it suitable for CPU execution.
The resulting feature map is highly discriminative and is processed by a prediction head for classification and bounding box regression.
Experimental Setup
SiamTPN was trained on a comprehensive suite of datasets, including LaSOT, GOT-10k, COCO, and TrackingNet. Evaluation was conducted on specialized UAV benchmarks (UAV123) and general tracking datasets (LaSOT, GOT-10k, VOT2018, OTB). Furthermore, real-world field tests were performed using a physical multi-copter equipped with an embedded PC and a PTZ (pan-tilt-zoom) camera to validate the tracker's performance in dynamic environments.
Results and Discussion
SiamTPN demonstrates a superior trade-off between accuracy and speed compared to existing state-of-the-art trackers.
Key findings include:
- Efficiency: The ShuffleNetV2 variant achieves over 30 FPS on a single CPU core. When utilizing a GPU, the speed exceeds 100 FPS.
- Accuracy: On the LaSOT dataset, SiamTPN reached an AUC of 58.1%, outperforming several heavier GPU-based models. On UAV123, it achieved a precision score of 85.8, surpassing the SiamAttn tracker.
- Robustness: Field tests confirmed the tracker's ability to maintain a steady frame rate while following targets of varying sizes and through complex maneuvers.
The "Pooling Attention" mechanism proves that full-resolution attention is not a prerequisite for capturing global context. By fusing pyramid layers, SiamTPN recovers structural details and semantic depth often lost in simplified lightweight architectures.
Conclusion
SiamTPN provides a viable path for deploying advanced Transformer-based tracking on edge devices. While performance is still somewhat bounded by the choice of the CNN backbone, the architecture offers a significant leap in CPU-based tracking efficiency. Future research will explore more aggressive model compression and the extension of this framework to multi-object tracking (MOT) scenarios.
Summary updated