Cooperative Aerial Robot Inspection Challenge: A Benchmark for Heterogeneous Multi-UAV Planning and Lessons Learned

Muqing Cao, Thien-Minh Nguyen, Shenghai Yuan, Andreas Anastasiou, Angelos Zacharia, Savvas Papaioannou, Panayiotis Kolios, Christos G. Panayiotou, Marios M. Polycarpou, Xinhang Xu, Mingjie Zhang, Fei Gao, Boyu Zhou, Ben M. Chen, Lihua XiearXiv 2025 · DOI · arXiv

Introduction

Autonomous aerial inspection is a critical task for maintaining large-scale infrastructure, yet it typically relies on a "map-first, inspect-later" workflow. In many real-world scenarios, detailed 3D models are unavailable or outdated, requiring robots to map and inspect simultaneously. The Cooperative Aerial Robot Inspection Challenge (CARIC) introduces a simulation-based benchmark to address this. It focuses on heterogeneous multi-UAV systems, pairing high-cost "Explorers" (LiDAR-equipped) with low-cost "Photographers" (camera-only) to perform high-quality structural inspections of complex assets without prior models.

Methods and System Architecture

The CARIC software stack is built on the Gazebo simulator and RotorS framework, providing a standardized arena for multi-agent coordination.

Heterogeneous Roles

The framework defines two distinct drone types to balance cost and capability:

  • Explorers: Larger UAVs equipped with LiDAR and cameras, responsible for generating 3D point clouds of the environment.
  • Photographers: Smaller, agile UAVs with gimbaled cameras designed for high-resolution data capture.
The CARIC framework utilizes heterogeneous UAVs: the LiDAR-equipped Explorer and the camera-focused Photographer.

Evaluation Metrics

Unlike simple coverage metrics, CARIC evaluates teams based on Visual Inspection Scores. These scores account for realistic constraints such as Line-of-Sight (LOS) obstructions, motion blur (penalizing high speeds during capture), and spatial resolution (requiring proximity to the target). Furthermore, the decentralized nature of the challenge forces algorithms to handle intermittent connectivity, as UAVs can only communicate when they have a clear line-of-sight to one another.

The CARIC Benchmark Scenarios

The benchmark includes several distinct simulated environments designed to test different aspects of multi-UAV planning:

  1. MBS: Inspection of the ArtScience Museum, featuring complex curved architecture.
  2. Hangar: A large indoor-outdoor transition environment containing an aircraft.
  3. Crane: A highly complex industrial structure with thin, lattice-like components that are difficult to map.
The CARIC benchmark includes diverse assets such as the Burj, Hangar, MBS, Crane, and Sail models.

Key Insights and Competition Results

Analysis of the top strategies from the CDC 2023 competition revealed several critical trade-offs in multi-UAV coordination:

  • Exploration vs. Inspection: The winning team, KIOS CoE, employed a "rapid exploration" strategy to map the area quickly. While effective for general structures, this approach struggled with the "Crane" scenario, where fast scans missed thin structural details.
  • Coordination Strategies: Team XXH utilized a "buddy system," grouping one explorer with several photographers. This ensured photographers always had local map updates, proving superior for intricate structures like cranes, where they captured 64% more interest points than the overall winner.
  • Motion Quality: High scores were strongly correlated with "stop-and-scan" behaviors and smooth trajectory optimization, rather than simply flying past interest points.
  • Robustness: Significant performance variance was observed due to pathfinding bugs or collisions in tight spaces, highlighting the need for more robust decentralized collision avoidance.

Conclusion and Future Directions

CARIC demonstrates that while global mapping followed by distributed task allocation is effective, specialized coordination is needed for complex geometries. Future work will focus on "communication-aware" planning to handle long-term LOS breaks, dynamic environments with moving obstacles, and stricter energy constraints to reflect real-world battery limitations.

Summary updated

Abstract

We propose the Cooperative Aerial Robot Inspection Challenge (CARIC), a simulation-based benchmark for motion planning algorithms in heterogeneous multi-UAV systems. CARIC features UAV teams with complementary sensors, realistic constraints, and evaluation metrics prioritizing inspection quality and efficiency. It offers a ready-to-use perception-control software stack and diverse scenarios to support the development and evaluation of task allocation and motion planning algorithms. Competitions using CARIC were held at IEEE CDC 2023 and the IROS 2024 Workshop on Multi-Robot Perception and Navigation, attracting innovative solutions from research teams worldwide. This paper examines the top three teams from CDC 2023, analyzing their exploration, inspection, and task allocation strategies while drawing insights into their performance across scenarios. The results highlight the task's complexity and suggest promising directions for future research in cooperative multi-UAV systems.
BibTeX
@misc{cao2025cooperative,
  author = {Muqing Cao and Thien-Minh Nguyen and Shenghai Yuan and Andreas Anastasiou and Angelos Zacharia and Savvas Papaioannou and Panayiotis Kolios and Christos G. Panayiotou and Marios M. Polycarpou and Xinhang Xu and Mingjie Zhang and Fei Gao and Boyu Zhou and Ben M. Chen and Lihua Xie},
  title = {{Cooperative Aerial Robot Inspection Challenge: A Benchmark for Heterogeneous Multi-UAV Planning and Lessons Learned}},
  year = {2025},
  eprint = {2501.06566},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.06566},
}
Figure