Mission-Level Robustness with Rapidly-Deployed, Autonomous Aerial Vehicles by Carnegie Mellon Team Tartan at MBZIRC 2020

Anish Bhattacharya, Akshit Gandhi, Lukas Merkle, Rohan Tiwari, Karun Warrior, Stanley Winata, Andrew Saba, Kevin Zhang, Oliver Kroemer, Sebastian Scherer

TL;DR

CMU’s Team Tartan demonstrates that "mission-level robustness"—the ability to detect and recover from subtask failures—is more critical for field robotics than the perfection of any single algorithm. By combining simple vision-based control with a robust state machine and an automated deployment pipeline, they achieved top-tier results at the MBZIRC 2020 competition without relying on expensive external positioning like RTK-GPS.

What Problem Does This Paper Address?

In the controlled environment of a lab, a robot might perform a task with 99% accuracy. However, in high-stakes, outdoor environments like the Mohamed Bin Zayed International Robotics Challenge (MBZIRC), that 1% failure rate is compounded across multiple mission stages (takeoff, search, detect, manipulate, land).

Most research focuses on lowering the failure rate of individual subtasks using complex, fragile algorithms. This paper argues for a different approach: Mission-level Robustness. The authors tackle the reality of field robotics—faulty hardware, unpredictable winds, and changing lighting—by designing a system that expects things to go wrong and encodes "recovery strategies" directly into the mission structure.

Methods

The team utilized a common hardware platform (DJI M210 V2) across three distinct challenges (balloon popping/ball catching, wall building, and firefighting). Their methodology rests on three pillars:

  1. Lightweight Attachments: Instead of heavy robotic arms, they used "UAV-friendly" tools, such as using the propellers themselves to pop balloons or a passive magnetic gripper for moving blocks.
  2. Vision Pipeline: They combined deep learning (TinyYOLOv3 via Intel OpenVINO) for general object detection with classical computer vision (HSV clustering and k-means) for high-precision tasks. For local interaction, they used Image-Based Visual Servoing (IBVS), which maps pixel errors directly to velocity commands, making the system less sensitive to camera calibration errors.
  3. Robust State Machines: Every mission was structured with "Outcome Monitoring." If a drone attempted to pick up a block and the sensor didn't detect a successful attachment, the state machine didn't just stop; it triggered a "Recover to Global Search" or "Re-attempt" loop.

Datasets

The authors relied on several data sources for their perception systems:

  • Custom Training Data: Hand-collected and augmented datasets of balloons and foam balls were used to train the YOLOv3 models to ensure they worked in both sunny and "blizzard-like" conditions.
  • On-site Adaptive Learning: For the construction challenge, they used a self-supervised approach. During an initial flight, the system used DBSCAN (a density-based clustering algorithm) to automatically learn the color and mask of the placement structure, allowing the robot to adapt to the specific lighting of the arena without manual labeling.

Key Insights and Ideas

  • The "Detect-Attempt-Recover" Cycle: The most successful missions weren't the ones that worked perfectly the first time, but the ones that failed, realized it, and tried again. For example, in the balloon-popping task, high-rate turns often caused the drone to lose sight of the target. The recovery logic allowed the drone to fly back to its last known "good" position and re-acquire the target autonomously.
  • Simplicity over Complexity: The team found that a simpler USB fisheye camera was often more reliable than the high-end gimbal camera, which suffered from software lag and mechanical offsets.
  • Rapid Deployment is a Feature: The team developed an automated "watchdog" pipeline that checked all sensors and launched ROS nodes upon power-up. This allowed them to complete seven mission runs in under 20 minutes during the Grand Challenge—a pace impossible for teams requiring manual SSH intervention.

Key Results

  • Challenge 1 (Aerial Objects): Successfully popped 5 balloons. While the "pop sequence" only had a 60% success rate on the first attempt, the recovery logic led to a 100% success rate for eventual completion.
  • Challenge 2 (Construction): Placed 4th. They were one of only three teams to successfully pick and place a block fully autonomously. Their IBVS system achieved 100% convergence during block alignment.
  • Challenge 3 (Firefighting): Dispensed the most water of any team onto a real outdoor fire target. Their system was efficient enough to complete eight full refill-and-extinguish cycles in a single 15-minute trial.
  • Grand Challenge: Placed 7th overall, despite persistent hardware issues, proving the resilience of their software architecture.

Limitations and Future Work

The authors identify several areas for improvement:

  • Monocular Limitations: Using a single camera made intercepting a moving target (the ball-catching task) extremely difficult. Future iterations would benefit from richer sensor input for 3D localization.
  • Open-Loop Drift: The final 0.7m of descent during block pickup was performed open-loop because the block obstructed the camera's view. This led to failures due to wind drift, which could be solved by using a wider-FOV fisheye camera mounted on the leg.
  • Multi-Agent Coordination: The current work focused on single-UAV robustness. The next step is scaling these robust state machines to coordinate multiple drones and ground vehicles (UGVs) simultaneously.

Summary updated

Abstract

For robotic systems to succeed in high risk, real-world situations, they have to be quickly deployable and robust to environmental changes, under-performing hardware, and mission subtask failures. These robots are often designed to consider a single sequence of mission events, with complex algorithms lowering individual subtask failure rates under some critical constraints. Our approach utilizes common techniques in vision and control, and encodes robustness into mission structure through outcome monitoring and recovery strategies. In addition, our system infrastructure enables rapid deployment and requires no central communication. This report also includes lessons in rapid field robotic development and testing. We developed and evaluated our systems through real-robot experiments at an outdoor test site in Pittsburgh, Pennsylvania, USA, as well as in the 2020 Mohamed Bin Zayed International Robotics Challenge. All competition trials were completed in fully autonomous mode without RTK-GPS. Our system placed fourth in Challenge 2 and seventh in the Grand Challenge, with notable achievements such as popping five balloons (Challenge 1), successfully picking and placing a block (Challenge 2), and dispensing the most water onto an outdoor, real fire with an autonomous UAV (Challenge 3).
BibTeX
@inproceedings{bhattacharya2022mission,
  author = {Anish Bhattacharya and Akshit Gandhi and Lukas Merkle and Rohan Tiwari and Karun Warrior and Stanley Winata and Andrew Saba and Kevin Zhang and Oliver Kroemer and Sebastian Scherer},
  title = {{Mission-Level Robustness with Rapidly-Deployed, Autonomous Aerial Vehicles by Carnegie Mellon Team Tartan at MBZIRC 2020}},
  booktitle = {Field},
  year = {2022},
  doi = {10.55417/fr.2022007},
}

Loading related content...

Figure