Abstract
Deep reinforcement learning has achieved great success in laser-based collision avoidance work because the laser can sense accurate depth information without too much redundant data, which can maintain the robustness of the algorithm when it is migrated from the simulation environment to the real world. However, high-cost laser devices are not only difficult to apply on a large scale but also have poor robustness to irregular objects, e.g., tables, chairs, shelves, etc. In this paper, we propose a vision-based collision avoidance framework to solve the challenging problem. Our method attempts to estimate the depth and incorporate the semantic information from RGB data to obtain a new form of data, pseudo-laser data, which combines the advantages of visual information and laser information. Compared to traditional laser data that only contains the one-dimensional distance information captured at a certain height, our proposed pseudo-laser data encodes the depth information and semantic information within the image, which makes our method more effective for irregular obstacles. Besides, we adaptively add noise to the laser data during the training stage to increase the robustness of our model in the real world, due to the estimated depth information is not accurate. Experimental results show that our framework achieves state-of-the-art performance in several unseen virtual and real-world scenarios.
Introduction
Traditional robot navigation often relies on 1D laser scanners (LiDAR). While fast and accurate, these sensors have a fatal flaw: they only "see" a thin horizontal slice of the world. If a laser is mounted 20cm off the ground, it might detect a table's legs but completely miss the overhanging tabletop, leading to a collision. Furthermore, sensors can be hampered by environmental factors like direct sunlight or non-reflective materials such as black metal.
On the other hand, using full RGB images for navigation introduces a massive "sim-to-real" gap—it is incredibly difficult to train a robot in a simulator and have its visual processing work perfectly in the messy real world. This paper seeks a middle ground: the richness of vision with the simplicity and robustness of laser-like data through a "pseudo-laser" representation.
Methodology
The proposed framework transforms a single RGB image into "pseudo-laser" data to help robots navigate around tricky, irregular obstacles. The pipeline consists of two primary stages: Environmental Perception and Control Decision.
Environmental Perception Stage
- Vision Processing: A single RGB image is passed through two parallel branches: FastDepth for depth estimation and CCNet for semantic segmentation.
- Semantic Masking: The framework uses the semantic map to identify "traversable regions" (like the floor). It then masks these out of the depth map so the robot doesn't mistake the ground for an obstacle.
- Deep Slicing (Dynamic Local Minimum Pooling): Instead of taking a single horizontal slice of the depth map, the authors perform column-wise minimum pooling on the bottom half of the image. This ensures that the "pseudo-laser" represents the closest obstacle in any given direction, regardless of its height.
Control Decision Stage
This 1D data is fed into a Deep Reinforcement Learning (DRL) model using the PPO algorithm. The architecture includes an Attention mechanism to focus on salient obstacles and an LSTM (Long Short-Term Memory) layer to help the robot "remember" objects that might have rotated out of its field of view. To bridge the sim-to-real gap, the authors adaptively add noise to the training data, emulating real-world depth estimation errors.
Experimental Results
The model was pre-trained on the NYU and KITTI datasets and evaluated in custom-built Gazebo and Stage simulation environments, as well as on a physical ROS-based mobile robot.
- Superiority over LiDAR: In complex scenes filled with irregular objects, traditional laser-based models had a success rate of only 15–21%. The proposed vision-based framework achieved an 89% success rate.
- Robustness to FOV: Even with a limited 90° field of view, the inclusion of LSTM and Attention allowed the model to maintain high success rates (94-100% in standard test scenes) by leveraging temporal memory.
- Sim-to-Real Transfer: The successful migration to a physical robot equipped with Intel RealSense cameras proved that the pseudo-laser representation effectively minimizes the sim-to-real gap by stripping away redundant visual data while preserving critical spatial information.
Conclusion
This framework provides a robust solution for navigating irregular environments where traditional LiDAR fails. By flattening 3D visual information into a 1D pseudo-laser safety map, the system achieves high-performance collision avoidance with a significantly reduced sim-to-real gap. Future work will address current limitations, such as performance on reflective surfaces like mirrors, and the integration of local avoidance with global positioning systems.
Summary updated