Improving Autonomous Robotic Navigation Using Imitation Learning

Brian Cesar-Tondreau, Garrett Warnell, Ethan Stump, Kevin Kochersberger, Nicholas R. WaytowichFrontiers Robotics AI 2021 · DOI

TL;DR

This paper introduces a novel approach to autonomous robot navigation by integrating imitation learning (IL) into existing navigation stacks. By inserting a machine learning module that learns from human demonstrations, the system can adapt its navigation behavior to user preferences without requiring extensive robotics expertise or code modification. This method demonstrates improved navigation success rates and trajectory similarity to human demonstrators compared to existing baselines.

What Problem Does This Paper Address?

Traditional autonomous navigation systems rely on a layered architecture of global and local path planning modules. While effective, modifying these systems to incorporate specific user preferences or adapt to dynamic environments is a complex and time-consuming process, often requiring significant expertise from roboticists. This paper addresses the challenge of making autonomous navigation more flexible and user-adaptable by enabling robots to learn desired navigation behaviors from human demonstrations. The goal is to simplify the process of customizing robot navigation for specific tasks or user styles, moving beyond the limitations of purely rule-based or optimization-driven approaches.

Methods

The core of the proposed method is the integration of a Behavioral Cloning (BC) module into a standard ROS navigation stack. This module is inserted between the global path planner and the local motion planner. Instead of replacing the entire navigation system, this learned module takes the current state (sensor data, goal information) and outputs a local goal for the existing local planner.

Key aspects of their method include:

  • Hybrid Approach: It leverages the robustness of classical navigation modules while incorporating the adaptability of machine learning.
  • Imitation Learning Module: A neural network is trained to map observed states to intermediate local goals.
  • Feature Extraction: The system uses a combination of lidar data, camera images (semantically segmented), distance to goal, and relative bearing to the goal as input features. Temporal stacking is used to provide historical context.
  • Action Space: The learned module predicts one of nine discrete "waypoint" classes, which are then passed to the local planner. This avoids the need for the learned model to handle low-level control signals directly.
  • Two-Phase Training:
    1. Pre-training: An initial policy is trained using demonstrations from the standard ROS navigation system (move_base) to learn basic navigation.
  • Human Update: This pre-trained policy is then fine-tuned using human demonstrations, allowing it to learn specific navigation styles or preferences.
  • Data Augmentation: To combat distributional shift common in BC, they employ a technique that synthetically generates additional training data by applying rotational transforms to the state and corresponding corrective actions.

Datasets

The paper describes experiments conducted in simulated environments.

  • Training Data:
    • Dnav: Demonstrations generated by the standard ROS move_base navigation system. These were collected by driving the robot between randomly generated start and end locations in simulated environments.
  • Dexpert: Demonstrations collected from a human tele-operating the robot. These were specifically designed to teach desired navigation styles, such as maintaining a wide berth around obstacles with a specific semantic label.
  • Evaluation Data:
    • The evaluation used novel navigation tasks (start/goal pairs) that were not part of the training set.
  • Experiments were conducted in both training and testing environments within two simulators: Gazebo and Unity.
  • A total of 12 additional task demonstrations (four trajectories per task) were used for evaluation, which were omitted from the training dataset.

Key Insights and Ideas

  • Bridging the Gap: The paper effectively demonstrates that integrating a learned module within an existing navigation stack, rather than replacing it entirely, is a powerful way to achieve adaptable navigation. This hybrid approach balances the guarantees of classical methods with the flexibility of learning.
  • Intuitive Customization: The use of human demonstrations for fine-tuning allows non-technical users to impart their navigation preferences, making autonomous systems more accessible and customizable.
  • Efficiency in Learning: The two-phase training strategy, combined with the specific action space and data augmentation, significantly reduces the amount of human demonstration data required compared to other IL approaches.
  • Generalization: The method shows promise in generalizing learned behaviors to unseen environments and tasks, a critical aspect for real-world deployment.

Key Results

The experimental results, visualized in Figure 4, show that the proposed method outperforms two recent baselines (Pfeiffer et al., 2018 and Bojarski et al., 2016) in both trajectory similarity to human demonstrations and successful navigation rate.

  • Trajectory Similarity: The proposed method achieved significantly higher similarity to human demonstrations across all tested environments (Gazebo and Unity, training and testing). This was measured using the inverse modified Hausdorff distance.
  • Successful Navigation Rate: The proposed method maintained a high task success rate in all four evaluation environments, outperforming the baselines.
  • Statistical Significance: Most experimental trials showed statistically significant improvements over the baselines, with the exception of the simplest Gazebo training environment, where all methods performed well.

Limitations and Future Work

  • Discrete Waypoint Limitation: The reliance on predicting discrete waypoints at a fixed distance limits the robot's ability to perform highly precise maneuvers that might be achievable by models predicting continuous control signals (e.g., wheel velocity).
  • Waypoint Generation Issues: The method sometimes generated waypoints within obstacle inflation bounds, leading to collisions and task failures. The authors suggest that explicit action-selection heuristics could mitigate this.
  • Future Work: The authors suggest exploring more precise maneuver capabilities and improving the robustness of waypoint generation. Further research into hyperparameter tuning for the neural network architecture is also mentioned.

Summary updated

BibTeX
@article{cesartondreau2021improving,
  author = {Brian Cesar-Tondreau and Garrett Warnell and Ethan Stump and Kevin Kochersberger and Nicholas R. Waytowich},
  title = {{Improving Autonomous Robotic Navigation Using Imitation Learning}},
  journal = {Frontiers Robotics AI},
  year = {2021},
  doi = {10.3389/frobt.2021.627730},
}
Figure