S2P2: Self-Supervised Goal-Directed Path Planning Using RGB-D Data for Robotic Wheelchairs

Hengli Wang, Yuxiang Sun, Rui Fan, Ming Liu

Introduction

Autonomous navigation for robotic wheelchairs is a critical technology for enhancing the mobility and independence of individuals with physical disabilities. Traditional navigation systems typically rely on a modular pipeline involving perception, path planning, and control. However, these systems often suffer from error propagation—where a small miscalculation in perception leads to unsafe path planning.

Recent shifts toward "end-to-end" deep learning models attempt to map sensor data directly to control signals, but they are frequently hampered by the "data bottleneck" of imitation learning, which requires extensive manual expert demonstrations. Furthermore, high-level commands like "turn left" are often too ambiguous for precise navigation in complex environments. The S2P2 (Self-Supervised Goal-Directed Path Planning) framework addresses these challenges by enabling wheelchairs to teach themselves how to reach specific goal coordinates using raw RGB-D data.

The S2P2 Framework

The core of S2P2 is a self-supervised learning loop that bridges the gap between traditional algorithmic planning and modern deep learning robustness. The framework is divided into two distinct phases: off-line training and on-line runtime.

Overview of the S2P2 framework, illustrating the off-line training phase using the Planned Path Generator (PPG) and the on-line runtime execution.

1. Off-line Training: The "Teacher"

To eliminate the need for human-labeled data, the authors developed the Planned Path Generator (PPG). The PPG acts as an automated expert. It processes RGB-D images to identify drivable areas through semantic segmentation and constructs a local costmap. Using traditional search algorithms like A or RRT, it calculates valid paths that serve as ground-truth labels for the neural networks.

2. On-line Runtime: The "Students"

The framework employs two specialized data-driven submodels to execute the planning:

  • Extrinsic 3-D Reconstruction Submodel (E3RS): This model treats path planning as a semantic segmentation task, generating a binary mask that highlights the intended path directly on the visual field.
  • Intrinsic 3-D Reconstruction Submodel (I3RS): This model utilizes a CNN-LSTM architecture to output a temporal sequence of coordinates, effectively modeling the path as a series of movements.
Architectural details of the Extrinsic (E3RS) and Intrinsic (I3RS) 3-D Reconstruction Submodels.

Technical Innovations

Best-Fit Regression Plane Loss ($L_{ER}$)

A significant contribution of this work is the introduction of a geometric constraint in the loss function. Recognizing that wheelchairs operate on a ground plane, the $L_{ER}$ loss penalizes predicted path points that deviate from the estimated ground surface (derived via disparity analysis). This forces the model to prioritize physically plausible, flat surfaces over impossible trajectories.

Robustness to Sensor Noise

One of the most compelling insights from the study is that deep learning models proved more resilient to "noisy" sensor data than the traditional algorithms used to train them. While traditional planners often failed entirely when perception systems made minor errors, the S2P2 models were able to infer safe paths by generalizing from their training data.

Key Results and Performance

The S2P2 framework was evaluated against traditional planners (A, JPS, PRM, RRT) and state-of-the-art benchmarks like the Risk-Contour Map (RCM).

  • Success Rates: S2P2 achieved a navigation success rate of approximately 89-90%, notably higher than the ~65-70% success rate of the traditional PPG algorithms used for its own training.
  • Real-World Validation: In live deployments, the system achieved a 95% success rate in indoor environments and 90% in outdoor settings, outperforming standard baselines such as the Dynamic Window Approach (DWA).
  • Path Quality: The LSTM-based I3RS model provided the lowest "Turning Cost," resulting in smoother and more comfortable trajectories for the wheelchair user.

Conclusion and Future Directions

S2P2 demonstrates that self-supervised learning can effectively bypass the data acquisition hurdles of imitation learning while providing superior robustness to environmental noise. While the current model excels in static environments, future work will focus on incorporating dynamic obstacle modeling to ensure safety in crowded spaces with moving pedestrians.

Summary updated

Abstract

Path planning is a fundamental capability for autonomous navigation of robotic wheelchairs. With the impressive development of deep-learning technologies, imitation learning-based path planning approaches have achieved effective results in recent years. However, the disadvantages of these approaches are twofold: 1) they may need extensive time and labor to record expert demonstrations as training data; and 2) existing approaches could only receive high-level commands, such as turning left/right. These commands could be less sufficient for the navigation of mobile robots (e.g., robotic wheelchairs), which usually require exact poses of goals. We contribute a solution to this problem by proposing S2P2, a self-supervised goal-directed path planning approach. Specifically, we develop a pipeline to automatically generate planned path labels given as input RGB-D images and poses of goals. Then, we present a best-fit regression plane loss to train our data-driven path planning model based on the generated labels. Our S2P2 does not need pre-built maps, but it can be integrated into existing map-based navigation systems through our framework. Experimental results show that our S2P2 outperforms traditional path planning algorithms, and increases the robustness of existing map-based navigation systems. Our project page is available at https://sites.google.com/view/s2p2.
BibTeX
@inproceedings{wang2021s,
  author = {Hengli Wang and Yuxiang Sun and Rui Fan and Ming Liu},
  title = {{S2P2: Self-Supervised Goal-Directed Path Planning Using RGB-D Data for Robotic Wheelchairs}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=M_B0QcHVi1K},
}
Figure