Visually Guided Balloon Popping with an Autonomous MAV at MBZIRC 2020

Marius Beul, Simon Bultmann, Andre Rochow, Radu Alexandru Rosu, Daniel Schleich, Malte Splietker, Sven Behnke

Abstract

Visually guided control of micro aerial vehicles (MAV) demands for robust real-time perception, fast trajectory generation, and a capable flight platform. We present a fully autonomous MAV that is able to pop balloons, relying only on onboard sensing and computing. The system is evaluated with real robot experiments during the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 where it showed its resilience and speed. In all three competition runs we were able to pop all five balloons in less than two minutes flight time with a single MAV.

Introduction

Autonomous aerial manipulation involves interacting with objects while in flight, a task that presents significant challenges in perception, localization, and physical stability. The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 task required MAVs to locate and pop five 60 cm diameter green balloons tethered to 2.5m poles within a 90m x 40m outdoor arena. Success in this environment requires robust perception against complex backgrounds, precise 3D localization, and a resilient physical design to withstand impacts.

System Design

The proposed system, nicknamed "Jelly," is built upon the DJI Matrice 100 platform. It integrates specialized hardware and a modular software architecture to achieve high-speed autonomous operation.

Hardware Configuration

The MAV is equipped with a variety of sensors and a custom popping mechanism. Key components include a GNSS for global positioning, an IMU for orientation, and a front-facing camera for target detection. A central onboard computer processes sensor data and runs the control algorithms, aided by a Google Edge TPU for hardware-accelerated vision.

The hardware components of the "Jelly" MAV, including the detachable tentacles and Edge TPU accelerator.

The most distinctive feature of Jelly is its popping mechanism: four 1.4m long flexible "tentacles" tipped with spiked hemispheres. These tentacles are designed to be detachable; if a rod becomes entangled with a balloon pole, it releases from the frame, allowing the MAV to continue its mission without crashing.

Software Architecture

The software stack is divided between the onboard computer and the flight controller. The onboard computer handles high-level tasks such as balloon detection, state estimation, and trajectory generation.

High-level system architecture showing the data flow from sensors to the MAV flight controller.

As shown in the architecture diagram, the perception pipeline feeds into a state machine, which then provides target coordinates to a trajectory generator. This generator produces roll, pitch, climb rate, and yaw rate commands at 50 Hz for the MAV's low-level flight controller.

Perception and Control

Vision Pipeline

The system utilizes a CNN-based detection model based on a modified ResNet-18 architecture for semantic segmentation. To meet real-time requirements, the network is quantized to 8-bit integers, enabling it to run on the Edge TPU with a latency of only 45ms per frame. 3D distance to the balloon is estimated geometrically by comparing the detected 2D radius in the image to the known physical size of the target (60cm).

Trajectory and Strategy

A Model Predictive Controller (MPC) generates time-optimal trajectories, limiting jerk and acceleration to ensure smooth approaches. A key strategic insight was the "Star-Shaped" approach: rather than flying directly between balloons, the MAV returns toward the center of the arena after each pop. This ensures a straight-line approach, preventing the dangling tentacles from swinging away from the target during turns.

Results

During the MBZIRC 2020 Grand Challenge, Jelly demonstrated exceptional performance, popping all five balloons in just 1 minute and 21 seconds. The vision pipeline successfully detected targets at ranges up to 50 meters, and the flexible tentacle design allowed for a 100% success rate on the first attempt for every balloon. The team secured 2nd place in the Grand Challenge and 5th in the balloon-popping sub-challenge.

Conclusion

The "Jelly" MAV demonstrates that combining specialized flexible hardware with accelerated onboard perception enables robust and high-speed autonomous interaction. Future work will address navigation in non-convex environments and extending the system to intercept moving targets.

Summary updated

Abstract

Visually guided control of micro aerial vehicles (MAV) demands for robust real-time perception, fast trajectory generation, and a capable flight platform. We present a fully autonomous MAV that is able to pop balloons, relying only on onboard sensing and computing. The system is evaluated with real robot experiments during the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 where it showed its resilience and speed. In all three competition runs we were able to pop all five balloons in less than two minutes flight time with a single MAV.
BibTeX
@inproceedings{beul2020visually,
  author = {Marius Beul and Simon Bultmann and Andre Rochow and Radu Alexandru Rosu and Daniel Schleich and Malte Splietker and Sven Behnke},
  title = {{Visually Guided Balloon Popping with an Autonomous MAV at MBZIRC 2020}},
  booktitle = {SSRR},
  year = {2020},
  url = {https://openreview.net/forum?id=2YCqWHbteY0},
}
Figure