Introduction
Navigating a robot through a crowded room with moving obstacles is a classic challenge in robotics. Traditional model-based approaches often require explicit mapping and object tracking, which are computationally expensive and prone to latency. Conversely, end-to-end deep learning methods often struggle with the "reality gap"—the difference between perfect simulation data and noisy real-world sensor input—and suffer from extreme sample inefficiency.
This paper introduces a decoupled learning pipeline that enables quadrupedal robots to navigate cluttered, dynamic environments using only a single depth camera. By separating unsupervised state representation learning from reinforcement learning (RL) policy training, the authors achieved a sample-efficient system that trains in just 12 minutes and transfers seamlessly from simulation to the real world.
Methods
The proposed modular pipeline is divided into three main stages: Encoding, Temporal Fusion, and Policy Learning. This architecture allows the system to build a memory-enabled representation of the world that remains consistent across both simulation and reality.
- Encoding (VAE): A Convolutional Variational Autoencoder (VAE) compresses raw depth images into a low-dimensional "latent vector." To bridge the reality gap, the authors apply a specific filtering process (dilation and hole closing) to both real and simulated images. This process blurs the distinction between the two, forcing the network to focus on general obstacle geometry rather than noisy details.
- Temporal Fusion (LSTM): Since a single camera frame provides only a snapshot, the robot requires memory to understand motion. An LSTM network processes the sequence of latent vectors and the camera's trajectory to form a "belief state" ($h_t$). This module is trained to predict future depth frames based on current movement.
- Policy Learning (RL): Finally, a Multi-Layer Perceptron (MLP) policy is trained using Proximal Policy Optimization (PPO). The policy operates solely on the compact "belief state" from the LSTM and the relative coordinates of the goal, rather than raw images.
Datasets and Evaluation
The system was trained using a combination of synthetic and real-world data:
- Simulation: 50,000 images were collected in NVIDIA’s Isaac Gym across various procedurally generated environments.
- Real-World: 20,000 images were gathered from different deployments of the ANYmal quadruped.
- Trajectories: 14 hours of simulated experience and 3 hours of real-world trajectories were used to train the temporal (LSTM) model.
- Deployment: The system was tested on 1,000 unseen simulated environments and deployed on a physical ANYmal robot equipped with an Intel RealSense D435 camera and a Jetson AGX Xavier.
Key Insights
- Decoupling for Efficiency: By training the vision representation separately from the driving policy, the RL stage becomes significantly faster. The navigation policy required only ~12 minutes of training.
- Intentional Information Loss: Using a small latent space and aggressive depth filtering improved robustness. Removing high-frequency noise made the model more effective when transitioning from simulation to the real world.
- Spatial Awareness through Memory: The LSTM allows the robot to maintain local spatial awareness. If an obstacle moves out of the camera's field of view, the LSTM's hidden state retains that information, preventing collisions with recently passed objects.
Results
The proposed "seq-to-seq" model demonstrated superior performance in both static and dynamic environments:
- Static Scenes: Achieved a 3% failure rate, significantly outperforming purely reactive policies (13% failure rate).
- Dynamic Environments: The failure rate was 11%, compared to 25% for reactive baselines.
- Computational Efficiency: The method was 2 to 3 times faster per learning iteration compared to end-to-end CNN+LSTM baselines.
- Sim-to-Real Transfer: The policy transferred to the physical ANYmal robot without additional fine-tuning, successfully navigating labs and outdoor spaces while avoiding pedestrians.
Conclusion and Future Work
While the system excels at local navigation, it remains "shortsighted," occasionally forgetting objects that stay out of view for extended periods. Future work aims to integrate this local planner with a higher-level global path planner for complex multi-room navigation and to extend the system's capabilities to rough, uneven terrain.
Summary updated