Abstract
Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. However, existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. In this work, we propose a simple yet effective framework, TransVisDrone, that provides an end-to-end solution with higher computational efficiency. We utilize CSPDarkNet-53 network to learn object-related spatial features and VideoSwin model to improve drone detection in challenging scenarios by learning spatio-temporal dependencies of drone motion. Our method achieves state-of-the-art performance on three challenging real-world datasets (Average [email protected]): NPS 0.95, FLDrones 0.75, and AOT 0.80, and a higher throughput than previous methods. We also demonstrate its deployment capability on edge devices and its usefulness in detecting drone-collision (encounter).
Introduction
Drone-to-drone (D2D) detection is a critical capability for collision avoidance, autonomous flight coordination, and security. However, detecting drones from the perspective of another moving platform is significantly harder than standard object detection. Target drones often occupy as little as 0.07% of the total frame area—essentially a few pixels. Furthermore, erratic motion from both the target and the source drone leads to motion blur and unpredictable trajectories. These challenges are compounded by complex backgrounds, such as cluttered landscapes, clouds, or abrupt illumination changes.
Previous state-of-the-art methods, such as DogFight, relied on complex, multi-stage pipelines involving frame patching and manual tracking. These non-differentiable components are difficult to train and often too slow for the real-time constraints of edge computing. TransVisDrone addresses these issues with a novel, end-to-end spatio-temporal transformer framework designed for high accuracy and real-time throughput.
Methodology
TransVisDrone is a simplified, fully differentiable architecture that processes video clips to leverage temporal context.
Spatial Feature Extraction
The model utilizes CSPDarkNet-53 (the backbone of YOLOv5) to extract multi-scale spatial features ($P_3, P_4, P_5$). This multi-resolution approach is vital for capturing the details of "tiny" objects that might otherwise be lost in deeper layers.
Spatio-Temporal Transformer
The core innovation is the integration of VideoSwin Transformer blocks. Unlike standard transformers that use global self-attention—which is computationally prohibitive for video—VideoSwin uses shifted 3D windows. This allows the model to learn temporal dependencies (how a drone moves across frames) without a massive increase in compute.
Training and Augmentation
To ensure the model learns actual object motion rather than processing artifacts, the authors use Temporally-Consistent Augmentation (TCA). This applies identical random augmentations (e.g., color jitter, perspective shifts) to every frame in a sampled clip. The final features are passed through a standard detection head and Non-Maximum Suppression (NMS) to produce bounding box sequences.
Datasets
The model was evaluated on three major benchmarks:
- NPS-Drones: 50 videos of custom delta-wing drones captured with GoPros.
- FL-Drones: A dataset featuring a mix of indoor and outdoor drone footage.
- Airborne Object Tracking (AOT): A large-scale, high-resolution dataset from Amazon Prime Air containing over 5.9 million frames, featuring diverse airborne objects like planes, birds, and helicopters.
Results and Discussion
TransVisDrone achieves state-of-the-art performance across all tested datasets. It achieved 0.95 AP on NPS and 0.80 AP on the AOT dataset, outperforming the previous best method (DogFight) by 6% on AOT.
Efficiency and Real-Time Performance
A key finding was the resolution trade-off: reducing resolution from 1280p to 640p resulted in only a 3% drop in Average Precision (AP) but a massive 250% increase in throughput. On a high-end NVIDIA RTX A6000, the model reaches 24.6 FPS. Crucially, on an NVIDIA Jetson Xavier NX edge board, it achieves a real-time 33 FPS at 640p resolution.
The Importance of Temporal Context
Single-frame detectors often fail because tiny drones are indistinguishable from background noise. By analyzing a short temporal window (3–5 frames), TransVisDrone distinguishes drones by their consistent trajectory and motion patterns. The model successfully detected drones in 46% of "closed encounter" flights, marking a significant step toward reliable autonomous collision avoidance.
Conclusion
TransVisDrone provides a robust, end-to-end solution for real-time drone-to-drone detection. While the current model focuses on detection, future work could extend these spatio-temporal features to multi-object tracking or monocular distance estimation. Reducing false positives in extremely cluttered "below-the-horizon" scenarios remains a priority for high-stakes aviation applications.
Summary updated