Laser Based Navigation in Asymmetry and Complex Environment

Yuchen Zhao, Keying Xie, Qingfei Liu, Yawen Li, Tian Wu

TL;DR

This paper introduces a novel laser-based navigation system for mobile robots that combines a global path planner with a deep reinforcement learning (DRL) based local planner. This hybrid approach significantly improves navigation efficiency and robustness in complex, asymmetric environments, outperforming existing end-to-end DRL methods and demonstrating strong generalization capabilities.

What Problem Does This Paper Address?

Navigating mobile robots in unstructured and cluttered environments is a significant challenge. While deep reinforcement learning (DRL) has shown promise in adapting to new environments with minimal human intervention, it often suffers from data inefficiency and a lack of robustness, particularly in asymmetric scenarios. Existing DRL methods can get stuck in "deadlock points" and may produce non-robust solutions susceptible to noise. Traditional planning methods, while robust, can be computationally intensive and require precise environmental modeling. This paper aims to bridge this gap by developing a navigation system that leverages the strengths of both global planning and DRL for more efficient, robust, and adaptable navigation.

Methods

The core of the proposed system is a hybrid approach combining a global planner with a DRL-based local planner.

  • Global Planner: An A* algorithm is used for global path planning, followed by a trajectory smoother. This planner provides a sequence of waypoints to guide the robot.
  • DRL Local Planner: Proximal Policy Optimization (PPO), an on-policy DRL algorithm, is employed for local trajectory re-planning. To address data efficiency and training speed, PPO is implemented asynchronously. This involves multiple "workers" collecting data in parallel from different environment scenarios, feeding into a central training process.
  • State Representation: The robot's state is represented by a combination of laser scanner observations, the local target (derived from the global planner), and the previous velocity command.
  • Reward Function: A carefully designed reward function encourages reaching the target, penalizes collisions, and rewards progress towards the local goal and path.

Datasets

The paper does not explicitly mention using pre-existing public datasets for training or evaluation. Instead, it describes experiments conducted in a virtual environment built using the EwayOS framework and a real robot platform named Tyran. The experiments involve various simulated scenarios, including typical office environments, harder tasks with challenging initial poses, and environments with unseen obstacles.

Key Insights and Ideas

  • Hybrid Planning is Key: Combining a global planner with a DRL-based local planner is more effective than purely end-to-end DRL for complex navigation tasks. The global planner provides a robust overall direction, while the DRL local planner handles immediate obstacle avoidance and trajectory adjustments.
  • Asynchronous PPO for Efficiency: Using asynchronous PPO significantly speeds up the training process by allowing parallel data collection, making DRL more practical for real-world applications.
  • Generalization with Few Shots: The proposed method demonstrates impressive generalization capabilities, able to adapt to unseen environments and obstacles with minimal additional training ("few shots").
  • Overcoming Deadlocks: The integration of a global planner helps the robot avoid getting trapped in deadlock situations, a common issue with purely local DRL navigators.
  • Asymmetric Performance Improvement: The system shows improved performance in asymmetric environments, where traditional methods might struggle.

Key Results

  • Superior Performance: The proposed APPO method achieved significantly higher total rewards compared to end-to-end DRL and ADDPG methods during training.
  • High Success Rates: In various testing scenarios, including hard modes and with unseen obstacles, the APPO method consistently achieved high success rates, often exceeding 90% and reaching up to 98-99% in some tasks.
  • Faster Adaptation: When adapting to a new scene, the model pre-trained on a previous scene learned significantly faster than a model trained from scratch, converging in half the time.
  • Robustness: The system demonstrated robustness in handling unseen obstacles, with only a moderate drop in success rate.

Limitations and Future Work

  • Dynamic Obstacles: The current study focuses on static obstacles and kinematic simulations. Future work should investigate the system's performance with dynamic obstacles and in more realistic dynamic models.
  • Integrated Learning: The paper suggests combining mapping, positioning, and navigation into a single learning model for a more holistic solution.
  • Transfer and Meta-Learning: Exploring transfer learning and meta-learning techniques could further enhance zero-shot or few-shot adaptation capabilities.
  • Rotation Behavior: A dedicated training scheme is needed to improve the agent's ability to handle difficult rotation behaviors, especially when escaping narrow paths with an unfavorable initial heading.
  • Real-World Deployment: While tested on a real robot platform, further extensive testing in diverse real-world environments is recommended.
  • Standalone Recovery Plug-in: The proposed method could be trained as a standalone recovery plug-in for existing local planners.

Summary updated

Abstract

For collision-free navigation in unstructured and cluttered environments, deep reinforcement learning (DRL) has gained extensive successes for being capable of adapting to new environments without much human effort. However, due to its asymmetry, the problems related to its lack of data efficiency and robustness remain as challenges. In this paper, we present a new laser-based navigation system for mobile robots, which combines a global planner with reinforcement learning-based local trajectory re-planning. The proposed method uses Proximal Policy Optimization to learn an efficient and robust local planning policy with asynchronous data generation and training. Extensive experiments have been presented, showing that the proposed system achieves better performance than previous methods including end-to-end DRL, and it can improve the asymmetrical performance. Our analysis show that the proposed method can efficiently avoid deadlock points and achieves a higher success rate. Moreover, we show that our system can generalize to unseen environments and obstacles with only a few shots. The model enables the warehouse to realize automatic management through intelligent sorting and handling, and it is suitable for various customized application scenarios.
BibTeX
@inproceedings{zhao2022laser,
  author = {Yuchen Zhao and Keying Xie and Qingfei Liu and Yawen Li and Tian Wu},
  title = {{Laser Based Navigation in Asymmetry and Complex Environment}},
  booktitle = {Symmetry},
  year = {2022},
  url = {https://openreview.net/forum?id=dEeyWVWRpM},
}
Figure