TL;DR
CMU’s Team Tartan demonstrates that "mission-level robustness"—the ability to detect and recover from subtask failures—is more critical for field robotics than the perfection of any single algorithm. By combining simple vision-based control with a robust state machine and an automated deployment pipeline, they achieved top-tier results at the MBZIRC 2020 competition without relying on expensive external positioning like RTK-GPS.
What Problem Does This Paper Address?
In the controlled environment of a lab, a robot might perform a task with 99% accuracy. However, in high-stakes, outdoor environments like the Mohamed Bin Zayed International Robotics Challenge (MBZIRC), that 1% failure rate is compounded across multiple mission stages (takeoff, search, detect, manipulate, land).
Most research focuses on lowering the failure rate of individual subtasks using complex, fragile algorithms. This paper argues for a different approach: Mission-level Robustness. The authors tackle the reality of field robotics—faulty hardware, unpredictable winds, and changing lighting—by designing a system that expects things to go wrong and encodes "recovery strategies" directly into the mission structure.
Methods
The team utilized a common hardware platform (DJI M210 V2) across three distinct challenges (balloon popping/ball catching, wall building, and firefighting). Their methodology rests on three pillars:
- Lightweight Attachments: Instead of heavy robotic arms, they used "UAV-friendly" tools, such as using the propellers themselves to pop balloons or a passive magnetic gripper for moving blocks.
- Vision Pipeline: They combined deep learning (TinyYOLOv3 via Intel OpenVINO) for general object detection with classical computer vision (HSV clustering and k-means) for high-precision tasks. For local interaction, they used Image-Based Visual Servoing (IBVS), which maps pixel errors directly to velocity commands, making the system less sensitive to camera calibration errors.
- Robust State Machines: Every mission was structured with "Outcome Monitoring." If a drone attempted to pick up a block and the sensor didn't detect a successful attachment, the state machine didn't just stop; it triggered a "Recover to Global Search" or "Re-attempt" loop.
Datasets
The authors relied on several data sources for their perception systems:
- Custom Training Data: Hand-collected and augmented datasets of balloons and foam balls were used to train the YOLOv3 models to ensure they worked in both sunny and "blizzard-like" conditions.
- On-site Adaptive Learning: For the construction challenge, they used a self-supervised approach. During an initial flight, the system used DBSCAN (a density-based clustering algorithm) to automatically learn the color and mask of the placement structure, allowing the robot to adapt to the specific lighting of the arena without manual labeling.
Key Insights and Ideas
- The "Detect-Attempt-Recover" Cycle: The most successful missions weren't the ones that worked perfectly the first time, but the ones that failed, realized it, and tried again. For example, in the balloon-popping task, high-rate turns often caused the drone to lose sight of the target. The recovery logic allowed the drone to fly back to its last known "good" position and re-acquire the target autonomously.
- Simplicity over Complexity: The team found that a simpler USB fisheye camera was often more reliable than the high-end gimbal camera, which suffered from software lag and mechanical offsets.
- Rapid Deployment is a Feature: The team developed an automated "watchdog" pipeline that checked all sensors and launched ROS nodes upon power-up. This allowed them to complete seven mission runs in under 20 minutes during the Grand Challenge—a pace impossible for teams requiring manual SSH intervention.
Key Results
- Challenge 1 (Aerial Objects): Successfully popped 5 balloons. While the "pop sequence" only had a 60% success rate on the first attempt, the recovery logic led to a 100% success rate for eventual completion.
- Challenge 2 (Construction): Placed 4th. They were one of only three teams to successfully pick and place a block fully autonomously. Their IBVS system achieved 100% convergence during block alignment.
- Challenge 3 (Firefighting): Dispensed the most water of any team onto a real outdoor fire target. Their system was efficient enough to complete eight full refill-and-extinguish cycles in a single 15-minute trial.
- Grand Challenge: Placed 7th overall, despite persistent hardware issues, proving the resilience of their software architecture.
Limitations and Future Work
The authors identify several areas for improvement:
- Monocular Limitations: Using a single camera made intercepting a moving target (the ball-catching task) extremely difficult. Future iterations would benefit from richer sensor input for 3D localization.
- Open-Loop Drift: The final 0.7m of descent during block pickup was performed open-loop because the block obstructed the camera's view. This led to failures due to wind drift, which could be solved by using a wider-FOV fisheye camera mounted on the leg.
- Multi-Agent Coordination: The current work focused on single-UAV robustness. The next step is scaling these robust state machines to coordinate multiple drones and ground vehicles (UGVs) simultaneously.
Summary updated