LaND: Learning to Navigate from Disengagements

Gregory Kahn, Pieter Abbeel, Sergey LevineIEEE RA-L 2020 · DOI · OpenReview

Abstract

Consistently testing autonomous mobile robots in real world scenarios is a necessary aspect of developing autonomous navigation systems. Each time the human safety monitor disengages the robot's autonomy system due to the robot performing an undesirable maneuver, the autonomy developers gain insight into how to improve the autonomy system. However, we believe that these disengagements not only show where the system fails, which is useful for troubleshooting, but also provide a direct learning signal by which the robot can learn to navigate. We present a reinforcement learning approach for learning to navigate from disengagements, or LaND. LaND learns a neural network model that predicts which actions lead to disengagements given the current sensory observation, and then at test time plans and executes actions that avoid disengagements. Our results demonstrate LaND can successfully learn to navigate in diverse, real world sidewalk environments, outperforming both imitation learning and reinforcement learning approaches.

Introduction

Developing autonomous mobile robots for complex real-world environments, such as sidewalks, presents significant challenges. Traditional approaches like Imitation Learning (IL) and Reinforcement Learning (RL) often struggle with generalization or safety during training. IL requires extensive expert demonstrations and can fail in out-of-distribution scenarios. RL, while capable of learning through trial and error, is often impractical in the real world due to the risk of collisions during the exploration phase.

Currently, developers use "disengagements"—instances where a human monitor intervenes to prevent a crash—primarily as a metric for system performance. LaND (Learning to Navigate from Disengagements) proposes a paradigm shift: using these interventions as the primary learning signal to train the navigation policy. By predicting which actions will lead to a human takeover, the robot can learn to navigate safely without explicit reward engineering.

Methodology

The core of LaND is a predictive model that estimates the probability of a human intervention given the current visual observation and a sequence of planned future actions. The model architecture combines a MobileNetV2 backbone for processing RGB camera images with an LSTM to handle temporal sequences of steering commands.

The LaND framework: During testing, the robot executes actions from the control policy while a human monitors. Disengagements are recorded. During training, this data is used to update a disengagement prediction model.

At test time, the robot employs Model Predictive Control (MPC). It samples thousands of potential trajectories and selects the one that minimizes the predicted probability of disengagement. This approach allows the robot to learn an implicit task specification—safety and correctness as defined by the human monitor—without the need for manually engineered reward functions.

Experimental Evaluation

The system was evaluated using a Clearpath Jackal robot equipped with an RGB camera. The researchers collected a dataset spanning 17.4 km of sidewalk navigation over 6 hours, capturing 1,926 unique disengagement events. This dataset includes diverse environmental factors such as varying lighting conditions, obstacles like bicycles and foliage, and complex geometry. Testing was conducted on 2.3 km of previously unseen sidewalks to evaluate generalization.

Results

LaND demonstrated superior performance compared to standard baselines. In terms of average distance between disengagements, LaND achieved 87.5 meters, significantly outperforming Behavioral Cloning (13.4 meters) and a state-of-the-art RL baseline (2.0 meters).

LaND enables autonomous navigation across diverse and challenging real-world sidewalk environments, as shown in these aerial and first-person trajectory views.

The system showed remarkable reliability, traveling 6.5 times further than the next best method before requiring intervention. In 33% of the test runs, LaND traveled more than 50 meters autonomously, with some runs exceeding 400 meters. Furthermore, finetuning the model on local data led to a performance increase from 101m to 218m per disengagement, highlighting the system's ability to scale with additional data.

Discussion and Future Work

LaND offers a "zero-cost" labeling mechanism by leveraging data that is already being collected during standard testing phases. It enables learning from failure safely by using the human monitor as a buffer. However, limitations remain. The current model does not explicitly learn from the corrective actions taken by the human after a disengagement. Future work could involve hybridizing LaND with imitation learning to utilize these expert corrections. Additionally, LaND could serve as a "patching" mechanism for existing navigation stacks, identifying and learning to avoid specific failure modes that are difficult to address with hand-coded logic.

Summary updated

Abstract

Consistently testing autonomous mobile robots in real world scenarios is a necessary aspect of developing autonomous navigation systems. Each time the human safety monitor disengages the robot's autonomy system due to the robot performing an undesirable maneuver, the autonomy developers gain insight into how to improve the autonomy system. However, we believe that these disengagements not only show where the system fails, which is useful for troubleshooting, but also provide a direct learning signal by which the robot can learn to navigate. We present a reinforcement learning approach for learning to navigate from disengagements, or LaND. LaND learns a neural network model that predicts which actions lead to disengagements given the current sensory observation, and then at test time plans and executes actions that avoid disengagements. Our results demonstrate LaND can successfully learn to navigate in diverse, real world sidewalk environments, outperforming both imitation learning and reinforcement learning approaches. Videos, code, and other material are available on our website https://sites.google.com/view/sidewalk-learning
BibTeX
@article{kahn2020land,
  author = {Gregory Kahn and Pieter Abbeel and Sergey Levine},
  title = {{LaND: Learning to Navigate from Disengagements}},
  journal = {IEEE Robotics and Automation Letters},
  year = {2020},
  doi = {10.1109/lra.2021.3060404},
}
Figure