Learning to fly by crashing

Dhiraj Gandhi, Lerrel Pinto, Abhinav Gupta

In the world of robotics, crashing is usually the ultimate failure—the thing engineers spend years trying to avoid. But what if the secret to robust navigation isn't avoiding failure, but embracing it? In their paper "Learning to Fly by Crashing," researchers from Carnegie Mellon University do exactly that: they "bite the bullet" and crash a drone 11,500 times to teach it how to fly.

TL;DR

Researchers developed a self-supervised learning approach for UAV navigation by deliberately crashing a drone into obstacles 11,500 times. By using these crashes as "negative labels," they trained a deep neural network that outperforms traditional depth-based methods and even rivals human performance in cluttered indoor environments.

What Problem Does This Paper Address?

Navigating an Unmanned Aerial Vehicle (UAV) through a cluttered indoor environment is a massive challenge for computer vision. Traditionally, researchers have tried three main paths, each with a significant bottleneck:

  1. Imitation Learning: Training on data from human experts. This is slow, expensive, and biased because humans rarely show the drone what a "near-crash" looks like.
  2. Simulation: Training in a virtual world. While safe, the "reality gap" is huge; simulations often fail to capture the nuances of real-world lighting, reflections, and textures.
  3. SLAM and Depth Sensing: Using sensors to map the world. These systems are often computationally heavy, expensive, and fail on tricky surfaces like glass doors or untextured white walls.

The authors argue that the primary reason we lack large-scale real-world data is the fear of crashing. By removing that fear, they can collect the massive amounts of data needed for high-capacity deep learning.

Methods

The team used a low-cost Parrot AR-Drone 2.0 equipped with a protective "safety hull." Their approach follows a simple but effective pipeline:

  • Autonomous Data Collection: The drone is placed in an environment, picks a random direction, and flies in a straight line until it hits something. An onboard accelerometer detects the spike of the impact.
  • Self-Supervised Labeling: The trajectory is automatically segmented. Images captured far from the crash are labeled as positive (safe to fly), while images captured just before the impact are labeled as negative (unsafe).
  • The Model: They used a standard AlexNet architecture pretrained on ImageNet. The final layer was modified into a binary softmax classifier to predict the probability of a crash.
  • The Flight Policy: At test time, the drone doesn't just look straight ahead. It takes three "crops" of its camera view (left, center, and right). If the center crop is safe, it flies forward. If the center is risky, it compares the left and right crops to decide which way to turn.

Datasets

The authors introduced a massive, first-of-its-kind UAV Crash Dataset:

  • Scale: 11,500 total crashes.
  • Flight Time: Over 40 hours of real-world flight data.
  • Diversity: Collected across 20 different indoor environments, including offices, labs, and corridors.

For evaluation, they tested the model in six specific environments, ranging from "Glass Doors" (notoriously hard for sensors) to a "Hallway with Chairs" (extremely cluttered).

Key Insights and Ideas

  • Negative Data is Gold: Most robotics datasets focus on "how to succeed." This paper proves that "how to fail" is equally, if not more, instructive for building robust systems.
  • Visual Cues over Geometry: Traditional depth-based systems struggle with glass because it’s transparent. However, a data-driven model can learn that "reflections on a flat surface" mean "don't go there," even if it doesn't understand the 3D geometry of the glass.
  • Simplicity Scales: By reducing navigation to a simple binary classification problem (Will I crash? Yes/No), the researchers were able to leverage powerful, well-understood deep learning architectures like AlexNet.
  • Surpassing Humans: In the "Hallway with Chairs" test, the drone actually outperformed human pilots. Humans often struggled with the drone's lag and the narrow gaps, whereas the model had "seen" those failure modes thousands of times.

Key Results

The model was compared against a "Best Straight Line" oracle, a state-of-the-art "Depth Prediction" baseline, and human pilots.

  • Superiority over Depth: In every single environment, the "Learning by Crashing" method significantly outperformed depth-based navigation. For example, in the NSH 4th Floor test, the model averaged 54.0 meters before a crash, compared to just 14.0 meters for the depth baseline.
  • The Glass Door Challenge: The model achieved an average distance of 27.9 meters in an area with transparent doors, while the depth-based model failed almost immediately (3.1 meters).
  • Robustness: The model successfully navigated for over 3 minutes continuously in a hallway, limited only by the drone's 5-minute battery life.

Limitations and Future Work

While groundbreaking, the paper notes several areas for growth:

  • Planar Flight: The current model primarily handles 2D navigation (left, right, forward). It does not yet account for vertical movement (up/down) or complex 3D maneuvers.
  • Hardware Durability: This method requires a drone that can survive 11,000 hits. This is feasible for a 400g plastic drone, but wouldn't work for a 10kg industrial UAV without significant hardware innovation.
  • Middle-Ground Data: The authors currently discard the "middle" part of the trajectory (between the safe start and the crash). Future work could look at using this data to learn more nuanced "near-miss" behaviors.

By proving that failure is a feature, not a bug, Gandhi, Pinto, and Gupta have provided a new blueprint for scaling up robot learning in the real world.

Summary updated

Abstract

How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. We show that this simple self-supervised model is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans. For supplementary video see:.
BibTeX
@inproceedings{gandhi2017learning,
  author = {Dhiraj Gandhi and Lerrel Pinto and Abhinav Gupta},
  title = {{Learning to fly by crashing}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2017},
  url = {https://openreview.net/forum?id=o93TLQPQzqn},
}

Loading related content...

Figure