Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation

Xinlei Pan, Tingnan Zhang, Brian Ichter, Aleksandra Faust, Jie Tan, Sehoon Ha

TL;DR

Researchers have developed a framework that allows legged robots to learn visual navigation by "watching" human demonstrations. By using a Feature Disentanglement Network (FDN) to bridge the perspective gap between a human-mounted camera and a robot's camera, the system achieves zero-shot imitation learning—enabling the robot to navigate to goals in environments it never traversed during training.

What Problem Does This Paper Address?

Collecting high-quality navigation data for legged robots is a nightmare. Unlike wheeled robots or self-driving cars, legged robots are often fragile, slow, and difficult to control for the thousands of hours required by standard deep learning.

To bypass this, researchers want to use imitation learning from humans. However, two massive hurdles exist:

  1. The Perspective Shift: A camera on a human’s chest sees the world differently than a camera mounted on a knee-high robot.
  2. Missing Labels: A video of a human walking doesn't tell the robot which motors to fire or which discrete "actions" (like "turn left") were taken.

Methods

The authors propose a multi-stage pipeline to turn human videos into robot policies:

  • Feature Disentanglement Network (FDN): This is the "secret sauce." It splits an image into two latent vectors: a state feature (the "where" and "what" of the scene) and a perspective feature (the "how it's viewed"). By using a cycle-loss (inspired by CycleGAN), the network learns to swap perspectives between images while keeping the underlying state constant. This allows the robot to interpret a human's goal image through its own "eyes."
  • Action Labeling: Since human videos lack robot action labels, the team used two methods. In simulation, they trained an Inverse Dynamics Model (IDM) to guess the action between two frames. In the real world, they used a custom GUI for rapid manual labeling.
  • Model-Based Imitation Learning: Based on the Universal Planning Network (UPN), the robot learns a transition model in the latent (hidden) feature space. To navigate, it performs gradient descent on its action sequence to minimize the distance between its predicted final state and the goal state provided by the human.

Datasets

The study utilized both simulated and real-world data:

  • NavWorld & OfficeWorld: Two PyBullet-based simulation environments. They collected 500 demonstration trajectories (20 steps each) using automated path-planning.
  • Real-World Laikago: Using the Laikago legged robot, they collected 25 human demonstration videos (about 10 minutes of footage) using three body-mounted GoPro cameras to provide multi-perspective data for the FDN.
  • Data Augmentation: They significantly boosted their dataset by replaying videos in reverse and adding "stay-in-place" sequences.

Key Insights and Ideas

  • Disentanglement is Better than Direct Transfer: Standard imitation learning (like UPN) fails miserably when the test perspective differs from the training perspective. FDN bridges this gap by creating a "perspective-invariant" state space.
  • Cycle-Loss vs. Triplet Loss: The authors found that their proposed cycle-loss was more stable and effective than the commonly used triplet loss. While triplet loss only learns the state feature, the cycle-loss forces the network to understand both state and perspective to successfully reconstruct images.
  • Zero-Shot Capability: The "zero-shot" nature here means the robot can navigate to a goal in a known environment using a path it has never seen or executed before, simply by planning in the learned latent space.

Key Results

  • Simulation: In the simpler NavWorld, the robot achieved a 79.66% success rate for short-distance tasks. In the more complex OfficeWorld, success was 78.71%.
  • Perspective Robustness: When tested against a baseline (UPN-PerspChange) that didn't use disentanglement, the authors' method was vastly superior. In NavWorld, the baseline success rate dropped to 7.88%, while this framework maintained 54.55%.
  • Real-World: On the physical Laikago robot, the system achieved a 60% success rate in navigating to targets roughly 2 meters away. The robot performed best when goals were visually distinct (like a brown chair) but struggled with low-contrast objects (like a white desk against a white wall).

Limitations and Future Work

The authors identify the manual labeling of human actions as the primary bottleneck for real-world scaling. While the IDM worked in simulation, it struggled with real-world robot data because legged gaits cause significant camera blur, making frame-to-frame transitions noisy.

Future work aims to:

  1. Integrate optical flow to automate human action labeling.
  2. Develop more stable gaits to reduce camera shake.
  3. Scale the system to use massive external datasets, such as Google Street View, to provide a more "universal" navigation intelligence.

Summary updated

Abstract

Imitation learning is a popular approach for training effective visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for long periods of time. In this work, we propose a zero-shot imitation learning framework for training a goal-driven visual navigation policy on a legged robot from human demonstrations (third-person perspective), allowing for high-quality navigation and cost-effective data collection. However, imitation learning from third-person demonstrations raises unique challenges. First, these demonstrations are captured from different camera perspectives, which we address via a feature disentanglement network (FDN) that extracts perspective-invariant state features. Second, as transition dynamics vary between systems, we reconstruct missing action labels by either building an inverse model of the robot's dynamics in the feature space and applying it to the human demonstrations or developing a Graphic User Interface (GUI) to label human demonstrations. To train a navigation policy we use a model-based imitation learning approach with FDN and action-labeled human demonstrations. We show that our framework can learn an effective policy for a legged robot, Laikago, from human demonstrations in both simulated and real-world environments. Our approach is zero-shot as the robot never navigates the same paths during training as those at testing time. We justify our framework by performing a comparative study.
BibTeX
@inproceedings{pan2020zero,
  author = {Xinlei Pan and Tingnan Zhang and Brian Ichter and Aleksandra Faust and Jie Tan and Sehoon Ha},
  title = {{Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  doi = {10.1109/icra40945.2020.9196602},
}
Figure