Multi-subgoal Robot Navigation in Crowds with History Information and Interactions

Xinyi Yu, Jianan Hu, Yuehai Fan, Wancai Zheng, Linlin Ou

Introduction

Navigating mobile robots through dense, dynamic crowds remains a significant challenge in robotics, often resulting in the "freezing robot" problem. This occurs when a robot, overwhelmed by the uncertainty of human movement, perceives all possible paths as unsafe and ceases motion. Traditional models often fail because they rely on instantaneous snapshots of the environment, ignoring the historical context of agent movement, or they output single velocity commands without a robust safety verification mechanism.

This paper presents SH-SACGNN, a hierarchical deep reinforcement learning framework designed to overcome these limitations by integrating history-aware trajectory encoding with a novel selection module for safe path planning.

Methodology

The SH-SACGNN framework employs a hierarchical structure to bridge high-level decision-making with low-level control.

The hierarchical control loop of SH-SACGNN, featuring the Deep RL Layer, Selection Signal, and Tracking Layer.

The architecture is built upon three core components:

1. Subgraph Network for History Encoding

To capture the intent and momentum of agents, the system vectorizes the trajectories of the robot and nearby humans over the preceding three time steps. A subgraph network processes these vectors to extract temporal features, providing the robot with a predictive understanding of the crowd's movement rather than just their current positions.

2. Graph Neural Network (GNN) with Self-Attention

Social interactions are modeled using a GNN where each agent is represented as a node. A self-attention mechanism allows the robot to dynamically weigh the importance of different agents. For instance, a pedestrian walking directly toward the robot is prioritized over one moving away, enabling more nuanced social navigation.

3. The Selection Module

A critical innovation is the selection module, which acts as a safety filter. Instead of generating a single action, the policy network samples multiple candidate subgoals. A dedicated selection network evaluates these candidates based on their expected long-term reward (Q-value), choosing the optimal and safest path.

Detailed neural network architecture showing the Subgraph Network, Graph Neural Network, and Selection Network components.

Experimental Evaluation

The model was evaluated using the CrowdNav simulation environment, specifically in "circle crossing" scenarios with 5 and 10 humans. These scenarios are designed to maximize potential collisions and test the robot's ability to find gaps in dense traffic. To validate real-world applicability, the system was also deployed on a SCOUT MINI mobile robot equipped with 16-line LiDAR.

Key Results

SH-SACGNN significantly outperformed baseline methods such as ORCA and RGL across several metrics:

  • Success Rate: Achieved a 98.4% success rate in 10-human scenarios, a substantial improvement over RGL's 88.6%.
  • Safety: The collision rate in 5-human scenarios was reduced to 0.6%, compared to 4.0% for RGL.
  • Robustness: The system maintained a 0% timeout rate, effectively solving the "freezing robot" problem by consistently identifying viable paths.

Discussion and Future Work

The inclusion of history information serves as a proxy for human intent, reducing hesitation. Furthermore, the sampling-based selection module provides a "second thought" mechanism that prevents the execution of aggressive or unreliable actions common in standard end-to-end models.

While highly effective, the system's average navigation time is slightly higher than more aggressive models, as it prioritizes safety. Future work will focus on optimizing navigation efficiency and extending the model to non-holonomic kinematic constraints, such as car-like steering, to support a wider variety of robotic platforms. Additionally, scaling the graph processing to handle even larger crowds remains a key area for development.

Summary updated

Abstract

Robot navigation in dynamic environments shared with humans is an important but challenging task, which suffers from performance deterioration as the crowd grows. In this paper, multi-subgoal robot navigation approach based on deep reinforcement learning is proposed, which can reason about more comprehensive relationships among all agents (robot and humans). Specifically, the next position point is planned for the robot by introducing history information and interactions in our work. Firstly, based on subgraph network, the history information of all agents is aggregated before encoding interactions through a graph neural network, so as to improve the ability of the robot to anticipate the future scenarios implicitly. Further consideration, in order to reduce the probability of unreliable next position points, the selection module is designed after policy network in the reinforcement learning framework. In addition, the next position point generated from the selection module satisfied the task requirements better than that obtained directly from the policy network. The experiments demonstrate that our approach outperforms state-of-the-art approaches in terms of both success rate and collision rate, especially in crowded human environments.
BibTeX
@misc{yu2022multi,
  author = {Xinyi Yu and Jianan Hu and Yuehai Fan and Wancai Zheng and Linlin Ou},
  title = {{Multi-subgoal Robot Navigation in Crowds with History Information and Interactions}},
  year = {2022},
  eprint = {2205.02003},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.02003},
}
Figure