Abstract
Despite significant advancements, autonomous driving systems continue to struggle with occluded objects and long-range detection due to the inherent limitations of single-perspective sensing. Aerial-ground cooperation offers a promising solution by integrating UAVs' aerial views with ground vehicles' local observations. However, progress in this emerging field has been hindered by the absence of public datasets and standardized evaluation benchmarks. To address this gap, this paper presents a comprehensive solution for aerial-ground cooperative 3D perception through three key contributions: (1) Griffin, a large-scale multi-modal dataset featuring over 200 dynamic scenes (30k+ frames) with varied UAV altitudes (20-60m), diverse weather conditions, and occlusion-aware 3D annotations, enhanced by CARLA-AirSim co-simulation for realistic UAV dynamics; (2) A unified benchmarking framework for aerial-ground cooperative detection and tracking tasks, including protocols for evaluating communication efficiency, latency tolerance, and altitude adaptability; (3) AGILE, an instance-level intermediate fusion baseline that dynamically aligns cross-view features through query-based interaction, achieving an advantageous balance between communication overhead and perception accuracy.
Introduction
While autonomous vehicles have come a long way, single-vehicle systems still struggle with "blind spots" caused by large obstacles or limited sensor range. Cooperative perception—where vehicles share data—is a promising solution. However, traditional Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) setups are expensive and slow to deploy.
Aerial-Ground Cooperation (AGC), which uses drones as "forward scouts" for ground vehicles, offers a flexible and cost-effective alternative. Despite its potential, the field has been stuck due to a lack of representative datasets. Existing simulations often use "ideal" conditions that don't translate to the messy reality of drone flight. Furthermore, many datasets lack occlusion-aware labels, leading to noisy training signals.
The Griffin Dataset and Methodology
The authors developed a modular co-simulation framework using CARLA (for traffic and environments) and AirSim (for high-fidelity drone physics and wind modeling). The resulting Griffin Dataset consists of 255 dynamic scenes totaling over 37,000 frames and 914,000 3D annotations across four distinct maps with varying weather, lighting, and wind speeds.
Key technical contributions include:
- Realistic Sensor Suites: The ground vehicle uses a multimodal setup (4 cameras + LiDAR), while the drone uses a vision-centric 5-camera array, reflecting real-world UAV constraints.
- Occlusion-Aware Annotation: A visibility quantification method ensures ground truth only includes objects visible from at least one perspective, preventing models from learning from "invisible" targets.
- Altitude Variations: Data is split into specific altitude bands (25m, 40m, 55m) and a "Random" set (20-60m) to test perspective shift handling.
Benchmarking and Insights
The paper evaluates four major fusion paradigms: Early Fusion (raw data), Intermediate Fusion (BEV-level or Instance-level), and Late Fusion (bounding boxes).
Key Insights
- The "Altitude Gap": As drones fly higher, ground targets shrink, making them harder to detect. Current models struggle to generalize across altitudes, highlighting a need for scale-aware architectures.
- Instance vs. Grid Fusion: Instance-level fusion (e.g., CoopTrack) proved more resilient than BEV-level grid fusion because transforming sparse 3D points is more robust to perspective distortions than aligning dense feature grids.
- Communication Efficiency: While Early Fusion provides the best accuracy (AP 0.607), it requires 311 MB/s. Intermediate fusion methods offer a "sweet spot," providing significant gains over single-vehicle perception while staying under 1 MB/s.
Robustness Thresholds
Cooperative methods significantly outperformed single-vehicle baselines, but they are sensitive to real-world noise:
- Latency: Performance drops below the "No Fusion" baseline once communication latency exceeds 200ms.
- Localization: If the drone's translation error exceeds 1.5 meters, the cooperation becomes a liability, performing worse than the vehicle acting alone.
Conclusion
Griffin provides a much-needed foundation for Aerial-Ground Cooperative perception. While the gains are significant, current systems remain fragile. Future research must focus on altitude-adaptive fusion, dynamic trust mechanisms to handle noisy data, and asynchronous processing to manage real-world timing discrepancies.
The dataset and code are available at: https://github.com/wang-jh18-SVM/Griffin
Summary updated