Introduction
Navigating mobile robots through dense, dynamic crowds remains a significant challenge in robotics, often resulting in the "freezing robot" problem. This occurs when a robot, overwhelmed by the uncertainty of human movement, perceives all possible paths as unsafe and ceases motion. Traditional models often fail because they rely on instantaneous snapshots of the environment, ignoring the historical context of agent movement, or they output single velocity commands without a robust safety verification mechanism.
This paper presents SH-SACGNN, a hierarchical deep reinforcement learning framework designed to overcome these limitations by integrating history-aware trajectory encoding with a novel selection module for safe path planning.
Methodology
The SH-SACGNN framework employs a hierarchical structure to bridge high-level decision-making with low-level control.
The architecture is built upon three core components:
1. Subgraph Network for History Encoding
To capture the intent and momentum of agents, the system vectorizes the trajectories of the robot and nearby humans over the preceding three time steps. A subgraph network processes these vectors to extract temporal features, providing the robot with a predictive understanding of the crowd's movement rather than just their current positions.
2. Graph Neural Network (GNN) with Self-Attention
Social interactions are modeled using a GNN where each agent is represented as a node. A self-attention mechanism allows the robot to dynamically weigh the importance of different agents. For instance, a pedestrian walking directly toward the robot is prioritized over one moving away, enabling more nuanced social navigation.
3. The Selection Module
A critical innovation is the selection module, which acts as a safety filter. Instead of generating a single action, the policy network samples multiple candidate subgoals. A dedicated selection network evaluates these candidates based on their expected long-term reward (Q-value), choosing the optimal and safest path.
Experimental Evaluation
The model was evaluated using the CrowdNav simulation environment, specifically in "circle crossing" scenarios with 5 and 10 humans. These scenarios are designed to maximize potential collisions and test the robot's ability to find gaps in dense traffic. To validate real-world applicability, the system was also deployed on a SCOUT MINI mobile robot equipped with 16-line LiDAR.
Key Results
SH-SACGNN significantly outperformed baseline methods such as ORCA and RGL across several metrics:
- Success Rate: Achieved a 98.4% success rate in 10-human scenarios, a substantial improvement over RGL's 88.6%.
- Safety: The collision rate in 5-human scenarios was reduced to 0.6%, compared to 4.0% for RGL.
- Robustness: The system maintained a 0% timeout rate, effectively solving the "freezing robot" problem by consistently identifying viable paths.
Discussion and Future Work
The inclusion of history information serves as a proxy for human intent, reducing hesitation. Furthermore, the sampling-based selection module provides a "second thought" mechanism that prevents the execution of aggressive or unreliable actions common in standard end-to-end models.
While highly effective, the system's average navigation time is slightly higher than more aggressive models, as it prioritizes safety. Future work will focus on optimizing navigation efficiency and extending the model to non-holonomic kinematic constraints, such as car-like steering, to support a wider variety of robotic platforms. Additionally, scaling the graph processing to handle even larger crowds remains a key area for development.
Summary updated