RELAX: Reinforcement Learning Enabled 2D-LiDAR Autonomous System for Parsimonious UAVs

Guanlin Wu, Zhuokai Zhao, Yutao He

Abstract

Unmanned Aerial Vehicles (UAVs) have gained significant prominence in recent years for areas including surveillance, search, rescue, and package delivery. One key aspect in UAV operations shared across all these tasks is the autonomous path planning, which enables UAV to navigate through complex, unknown, and dynamic environments while avoiding obstacles without human control. Despite countless efforts having been devoted to this subject, new challenges are constantly arisen due to the persistent trade-off between performance and cost. And new studies are more urgently needed to develop autonomous system for UAVs with parsimonious sensor setup, which is a major need for wider adoptions. To this end, we propose an end-to-end autonomous framework to enable UAVs with only one single 2D-LiDAR sensor to operate in unknown dynamic environments. More specifically, we break our approach into three stages: a pre-processing Map Constructor; an offline Mission Planner; and an online reinforcement learning (RL)-based Dynamic Obstacle Handler. Experiments show that our approach provides robust and reliable dynamic path planning and obstacle avoidance with only 1/10 of the cost in sensor configuration.

Introduction

Autonomous navigation for Unmanned Aerial Vehicles (UAVs) typically necessitates high-fidelity sensors such as 3D-LiDAR or RGB-D cameras. While these sensors provide rich environmental data, their high cost and computational requirements limit the accessibility of autonomous drones for widespread commercial and agricultural applications. The challenge lies in developing "parsimonious" UAVs—drones that can navigate safely using minimal, low-cost hardware.

A single 2D-LiDAR sensor, while inexpensive, presents significant hurdles. It only captures a horizontal slice of the environment, and its data becomes noisy or invalid when the drone tilts during flight. Furthermore, existing 2D-LiDAR solutions often require pre-mapped environments or fail in the presence of dynamic obstacles. The RELAX (Reinforcement Learning Enabled 2D-LiDAR Autonomous System) framework addresses these limitations by providing an end-to-end pipeline for autonomous flight in unknown, dynamic settings.

System Architecture

The RELAX framework is designed as a modular system comprising three primary components: the Map Constructor, the Mission Planner, and the Online Re-planner.

The RELAX framework architecture, illustrating the modular interaction between the Map Constructor, Mission Planner, and Online Re-planner.
  1. Map Constructor: This module utilizes the Hector-SLAM algorithm to transform raw 2D-LiDAR scans into a 2D occupancy grid map. This enables the UAV to build a representation of unknown environments in real-time.
  2. Mission Planner: Using the generated static map, a Rapidly-exploring Random Tree (RRT) algorithm calculates a collision-free global path. A "Point Transformer" then converts map-based pixel coordinates into real-world coordinates for the flight controller.
  3. Online Re-planner: This core component handles dynamic obstacles. It employs a Dueling Double Deep Q-Network (D3QN) to adjust the UAV's path in real-time. To mitigate sensor noise caused by drone tilt, a physics-informed heuristic filter discards LiDAR points that suggest physically impossible obstacle movements.

Reinforcement Learning for Dynamic Obstacle Avoidance

The D3QN agent is trained to make critical navigation decisions in high-risk states while maintaining efficiency in open spaces. The state space for the RL agent includes a target direction vector and 2D-LiDAR data segmented into eight circular sectors. By separating global path planning from local re-planning, the system remains computationally lightweight, with an inference time as low as 0.0003 seconds.

Experimental Evaluation

The system was rigorously tested in the ROS-Gazebo-PX4 simulation environment. The researchers designed complex scenarios, including an "agricultural farmland" environment featuring randomly distributed dynamic obstacles like iron bars.

The Gazebo simulation environment used for training and testing, showing the UAV navigating an agricultural-style environment with dynamic obstacles.

In 50 test runs with 20 dynamic obstacles, RELAX achieved a 90% success rate. This represents a significant improvement over previous state-of-the-art 2D-LiDAR RL methods, such as Gabriel’s algorithm, which achieved only a 10% success rate under identical conditions. While 3D-LiDAR systems (e.g., FAST-LIO) can reach a 98% success rate, RELAX offers comparable performance at approximately 1/10th of the hardware cost.

Conclusion

RELAX demonstrates that high-performance autonomous navigation is achievable with parsimonious sensor configurations. By combining traditional SLAM and RRT with advanced reinforcement learning, the framework bridges the gap between cost and capability. Future work will explore the integration of multiple 2D-LiDAR sensors at varying angles to provide a "pseudo-3D" field of view, further enhancing obstacle detection without the high cost of 3D-LiDAR hardware.

Summary updated

Abstract

Unmanned Aerial Vehicles (UAVs) have gained significant prominence in recent years for areas including surveillance, search, rescue, and package delivery. One key aspect in UAV operations shared across all these tasks is the autonomous path planning, which enables UAV to navigate through complex, unknown, and dynamic environments while avoiding obstacles without human control. Despite countless efforts having been devoted to this subject, new challenges are constantly arisen due to the persistent trade-off between performance and cost. And new studies are more urgently needed to develop autonomous system for UAVs with parsimonious sensor setup, which is a major need for wider adoptions. To this end, we propose an end-to-end autonomous framework to enable UAVs with only one single 2D-LiDAR sensor to operate in unknown dynamic environments. More specifically, we break our approach into three stages: a pre-processing Map Constructor; an offline Mission Planner; and an online reinforcement learning (RL)-based Dynamic Obstacle Handler. Experiments show that our approach provides robust and reliable dynamic path planning and obstacle avoidance with only 1/10 of the cost in sensor configuration. The code will be made public upon acceptance.
BibTeX
@misc{wu2023relax,
  author = {Guanlin Wu and Zhuokai Zhao and Yutao He},
  title = {{RELAX: Reinforcement Learning Enabled 2D-LiDAR Autonomous System for Parsimonious UAVs}},
  year = {2023},
  eprint = {2309.08095},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.08095},
}
Figure