Abstract
Consistently testing autonomous mobile robots in real world scenarios is a necessary aspect of developing autonomous navigation systems. Each time the human safety monitor disengages the robot's autonomy system due to the robot performing an undesirable maneuver, the autonomy developers gain insight into how to improve the autonomy system. However, we believe that these disengagements not only show where the system fails, which is useful for troubleshooting, but also provide a direct learning signal by which the robot can learn to navigate. We present a reinforcement learning approach for learning to navigate from disengagements, or LaND. LaND learns a neural network model that predicts which actions lead to disengagements given the current sensory observation, and then at test time plans and executes actions that avoid disengagements. Our results demonstrate LaND can successfully learn to navigate in diverse, real world sidewalk environments, outperforming both imitation learning and reinforcement learning approaches.
Introduction
Developing autonomous mobile robots for complex real-world environments, such as sidewalks, presents significant challenges. Traditional approaches like Imitation Learning (IL) and Reinforcement Learning (RL) often struggle with generalization or safety during training. IL requires extensive expert demonstrations and can fail in out-of-distribution scenarios. RL, while capable of learning through trial and error, is often impractical in the real world due to the risk of collisions during the exploration phase.
Currently, developers use "disengagements"—instances where a human monitor intervenes to prevent a crash—primarily as a metric for system performance. LaND (Learning to Navigate from Disengagements) proposes a paradigm shift: using these interventions as the primary learning signal to train the navigation policy. By predicting which actions will lead to a human takeover, the robot can learn to navigate safely without explicit reward engineering.
Methodology
The core of LaND is a predictive model that estimates the probability of a human intervention given the current visual observation and a sequence of planned future actions. The model architecture combines a MobileNetV2 backbone for processing RGB camera images with an LSTM to handle temporal sequences of steering commands.
At test time, the robot employs Model Predictive Control (MPC). It samples thousands of potential trajectories and selects the one that minimizes the predicted probability of disengagement. This approach allows the robot to learn an implicit task specification—safety and correctness as defined by the human monitor—without the need for manually engineered reward functions.
Experimental Evaluation
The system was evaluated using a Clearpath Jackal robot equipped with an RGB camera. The researchers collected a dataset spanning 17.4 km of sidewalk navigation over 6 hours, capturing 1,926 unique disengagement events. This dataset includes diverse environmental factors such as varying lighting conditions, obstacles like bicycles and foliage, and complex geometry. Testing was conducted on 2.3 km of previously unseen sidewalks to evaluate generalization.
Results
LaND demonstrated superior performance compared to standard baselines. In terms of average distance between disengagements, LaND achieved 87.5 meters, significantly outperforming Behavioral Cloning (13.4 meters) and a state-of-the-art RL baseline (2.0 meters).
The system showed remarkable reliability, traveling 6.5 times further than the next best method before requiring intervention. In 33% of the test runs, LaND traveled more than 50 meters autonomously, with some runs exceeding 400 meters. Furthermore, finetuning the model on local data led to a performance increase from 101m to 218m per disengagement, highlighting the system's ability to scale with additional data.
Discussion and Future Work
LaND offers a "zero-cost" labeling mechanism by leveraging data that is already being collected during standard testing phases. It enables learning from failure safely by using the human monitor as a buffer. However, limitations remain. The current model does not explicitly learn from the corrective actions taken by the human after a disengagement. Future work could involve hybridizing LaND with imitation learning to utilize these expert corrections. Additionally, LaND could serve as a "patching" mechanism for existing navigation stacks, identifying and learning to avoid specific failure modes that are difficult to address with hand-coded logic.
Summary updated