Abstract
Unmanned aerial vehicle (UAV)-based visual object tracking has enabled a wide range of applications in intelligent transportation systems. Siamese networks have emerged as a leading approach due to their balance of accuracy and speed. However, limited onboard computational resources and complex aerial environments pose significant challenges. This work presents a comprehensive review of state-of-the-art Siamese trackers, evaluates them on typical UAV onboard processors, and analyzes their performance in real-world scenarios, including low-illumination conditions.
Introduction: The Challenges of Aerial Tracking
Visual object tracking from a UAV platform is significantly more demanding than standard tracking tasks. Drones operate under a "triple threat" of constraints:
- Computational Scarcity: Onboard processors must be lightweight and power-efficient, unlike high-end server GPUs.
- Dynamic Environments: Rapid maneuvers and viewpoint changes lead to motion blur and scale variations.
- Environmental Factors: Aerial footage often suffers from low resolution and poor lighting, particularly during nighttime operations.
While Siamese networks are popular for their efficiency, their deployment on resource-constrained UAV hardware requires careful optimization to maintain real-time performance.
Methodology and Taxonomy
The Siamese tracking framework typically follows a dual-branch architecture consisting of a template branch and a search branch.
As shown in the figure above, the process involves feature extraction from both branches, followed by feature refinement and cross-correlation. The resulting response maps are then processed through classification and regression heads to localize the target. The authors categorize the evolution of these trackers into a clear taxonomy based on their architectural innovations:
- Feature Modeling: This includes "Pioneers" like SiamFC, innovations in feature extraction (e.g., deeper backbones), and feature refinement techniques (e.g., attention mechanisms in SiamAttn).
- Target Localization: Trackers are divided into Anchor-based methods (using predefined bounding boxes) and Anchor-free methods (directly regressing coordinates). Anchor-free models often prove more robust for the small, varying scales typical of UAV footage.
- Extra Types: Specialized models like LightTrack (using Neural Architecture Search) or GlobalTrack for long-term tracking.
Comprehensive Benchmarking
To bridge the gap between theoretical performance and real-world deployment, 19 state-of-the-art trackers were integrated into a unified library and tested on an NVIDIA Jetson AGX Xavier, a standard UAV onboard processor.
Key Insights
- The 30 FPS Barrier: For effective UAV control, a tracker must exceed 30 FPS. Many high-accuracy models fail this on embedded hardware.
- Efficiency vs. Depth: While deep backbones (ResNet-50) dominate desktop benchmarks, shallower architectures (like AlexNet-based SiamRPN++ or SiamAPN) often perform better on UAVs by preserving resources for flight-critical tasks.
- Anchor-Free Superiority: Models like SiamBAN and SiamAPN++ handle extreme scale variations better than rigid anchor-based systems.
- The Nighttime Gap: Performance often drops by over 40% in low-light conditions, highlighting a major area for improvement.
Results and Performance
SiamAPN and SiamAPN++ emerged as the top performers for UAV deployment, offering the best trade-off between precision and speed. On the Jetson AGX Xavier, SiamAPN achieved approximately 35 FPS with a precision score of 0.773, while the AlexNet-based SiamRPN++ reached a blistering 52 FPS.
Future Directions
The study identifies several critical areas for future research:
- Low-Light Robustness: Developing trackers that can handle noise and low contrast without heavy active sensors.
- Latency Compensation: Integrating Predictive Visual Tracking (PVT) to account for computational lag.
- Security: Addressing adversarial vulnerabilities where small image perturbations can cause tracking failure.
- Automated Design: Utilizing Neural Architecture Search (NAS) to optimize models specifically for embedded hardware constraints.
The unified code library and evaluation results are available at https://github.com/vision4robotics/SiameseTracking4UAV.
Summary updated