Abstract
Navigating dense, dynamic environments remains a significant challenge in robotics. While Deep Reinforcement Learning (DRL) offers potential for time-sequential decision-making, standard models often struggle with state representation and generalization. This paper introduces LSA-DSAC, a hybrid algorithm that enhances the Discrete Soft Actor-Critic (DSAC) framework with a sophisticated representation learning encoder. By combining attention mechanisms, skip connections, and LSTM pooling, the proposed method achieves a 99.6% success rate in complex simulations and demonstrates robust sim-to-real transferability.
Introduction
Robotic motion planning in crowded, unpredictable spaces—such as delivery bots in busy lobbies—requires a delicate balance between safety and efficiency. Traditional algorithms like A* are often restricted to static maps, while reactive methods like Optimal Reciprocal Collision Avoidance (ORCA) can be overly conservative or "myopic" at high speeds. DRL has emerged as a powerful alternative, but its performance is heavily dependent on the "expressive power" of the environmental state representation. Poor representations lead to slow convergence and overfitting to specific training scenarios.
Methodology: The LSA-DSAC Framework
The core contribution of this work is the LSA-DSAC algorithm, which optimizes the representation of obstacles through a four-step architectural evolution of the encoder:
- Attention-Weighted (AW) Encoding: Instead of treating all obstacles equally, an attention mechanism weighs obstacles based on their relative speed, distance, and direction.
- Skip Connections (SA): To prevent the loss of fine-grained environmental details during deep processing, the authors integrated skip connections. This allows the model to concatenate raw input features with high-level attention features, mitigating overfitting.
- LSTM Pooling (LSA): Traditional attention models often use simple summation to aggregate obstacle data. However, summation lacks "injectivity"—different obstacle configurations can result in the same sum. By replacing summation with Long Short-Term Memory (LSTM) pooling, the model preserves the structural and statistical nuances of the obstacle field.
- Discrete Soft Actor-Critic (DSAC): These enhanced representations feed into a DSAC framework, where the policy and critic use separate attention networks to improve convergence.
Experimental Evaluation and Results
The model was rigorously tested in both simulated and physical environments. Training was primarily conducted in a "Circle-crossing" simulator, where obstacles converge toward the center, creating high-density conflict zones.
Performance Metrics
LSA-DSAC demonstrated state-of-the-art performance across several key metrics:
- Success Rate: The model achieved a 99.6% success rate in 5-obstacle scenarios, outperforming SARL (99.0%) and vastly exceeding classical ORCA (43%).
- Safety: It recorded the lowest collision rate (0.004) among all tested algorithms.
- Generalization: In a "Square-crossing" transferability test, LSA-DSAC maintained a 93% success rate, proving its ability to handle unseen environmental geometries.
Real-World Validation
The system was deployed on a physical mobile robot equipped with a Jetson Nano and an optical motion capture system. The robot successfully navigated through both static and dynamic human obstacles, confirming that the LSA-DSAC model effectively bridges the "sim-to-real" gap.
Discussion and Conclusion
The success of LSA-DSAC highlights the importance of "injectivity" in feature aggregation and the role of skip connections in maintaining environmental context. While the increased complexity of the LSA encoder leads to slightly slower initial training convergence, the final performance gains in safety and success rate are substantial. Future work will focus on improving the interpretability of the attention weights and exploring Bayesian models to infer hidden obstacle intentions to further refine navigation safety.
Summary updated