Researchers from the Autonomous Intelligent Systems Group at the University of Bonn have developed a robust approach to a classic robotics challenge: enabling a humanoid robot to navigate unknown environments without a pre-existing map. By combining "imagination-based" planning with a novel failure prediction mechanism, the team successfully transferred a controller trained entirely in simulation to a real NimbRo-OP2X humanoid robot.
Abstract
This work introduces a mapless navigation system for humanoid robots that utilizes a learned world model to plan paths by imagining future outcomes in a compressed latent space. By integrating visual data with internal sensor measurements and a novel "termination likelihood" predictor, the system achieves high sample efficiency and successfully transfers skills from simulation to real-world hardware. The approach demonstrates an 86% success rate in complex simulated environments and real-time navigation on physical hardware.
Introduction
Navigation for humanoid robots is notoriously difficult. Unlike wheeled platforms, humanoids experience significant camera "bobbing" due to their gait, possess a high center of mass that makes them prone to falling, and require complex, high-dimensional movement control. Traditional navigation relies on building a map via SLAM (Simultaneous Localization and Mapping), which is computationally expensive and often fails in dynamic environments.
Mapless navigation using Deep Reinforcement Learning (DRL) offers a promising alternative but typically requires massive amounts of data. Furthermore, while most existing mapless systems rely on depth sensors like LiDAR, using only RGB cameras is significantly harder as the robot must infer distances and geometry from flat images.
Methods
The system builds upon the Dreamer architecture, a model-based RL framework. The core components include:
- Latent World Model: Instead of processing raw pixels for every decision, the system uses a Recurrent State Space Model (RSSM) to compress visual and sensor data into a "latent space." The robot then "imagines" future trajectories within this compressed space to plan actions.
- Sensor Fusion: The model fuses segmented RGB images with non-visual data, including gait velocity, head position, and IMU data (pitch and roll). This provides the agent with "self-awareness" regarding its stability.
- Termination Likelihood Predictor: A key contribution of this paper is an auxiliary loss function that predicts the likelihood of an episode ending in success (reaching the goal) or failure (falling). This helps the agent anticipate and avoid terminal states early in the learning process.
- Sim-to-Real Transfer: To bridge the gap between simulation and reality, the system employs semantic segmentation. By converting raw camera feeds into simplified, color-coded maps of "obstacle vs. floor," the robot ignores real-world textures or lighting changes that were absent in the simulator.
Key Results and Insights
The model was trained entirely in the MuJoCo physics engine using eight parallel simulated environments. Evaluation on 100 random simulated scenes showed an 86% success rate.
Ablation studies highlighted the importance of specific components:
- Predicting Failure: Without the termination predictor, the success rate dropped to 70%, proving that explicitly modeling failure improves sample efficiency.
- The Necessity of Proprioception: Without non-visual (IMU/gait) data, the success rate plummeted to 0%. For a humanoid, internal stability data is as critical as external visual data.
- Abstraction via Segmentation: Using segmented images allowed the robot to focus on task geometry rather than visual noise, facilitating a smooth sim-to-real transfer without retraining on real hardware.
Conclusion and Future Work
The system successfully enabled a real robot to navigate narrow passages and avoid moving obstacles in real-time (10 Hz inference). However, limitations remain. The agent can occasionally get stuck in local minima during complex "back-up" maneuvers and lacks a high-level hierarchical planner for long-distance navigation. Future work will focus on incorporating hierarchical policies and estimating the velocity of dynamic obstacles to allow for more proactive path planning.
Summary updated