CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning

Tianyu Li, Hyunyoung Jung, Matthew C. Gombolay, Yong Kwon Cho, Sehoon Ha

TL;DR

CrossLoco is a novel reinforcement learning framework that allows robots with completely different body structures—like a four-legged quadruped—to "mimic" human motions without requiring a pre-labeled dataset of matching movements. By using a cycle-consistency reward system, the framework simultaneously discovers robot skills and establishes a semantic bridge between human kinematics and robot dynamics.

What Problem Does This Paper Address?

Human Motion Driven Control (HMDC) is the "holy grail" of intuitive teleoperation: you move, and the robot follows. However, establishing a correspondence between a two-legged human and, say, a four-legged robot is a nightmare of "morphological mismatch."

Previous approaches usually fell into two camps:

  1. Supervised Learning: Requires massive, hand-labeled datasets where every human pose is paired with a corresponding robot pose. This is incredibly labor-intensive.
  2. Unsupervised Learning: Tries to find patterns without pairs but often fails on complex, high-dimensional robots because the search space is too vast.

The authors tackle the "intrinsic ambiguity" of the problem: there isn't just one way for a quadruped to "walk" like a human. How do you teach a robot to capture the essence of a human motion (like dancing or running) when its body is fundamentally different?

Methods

The core of CrossLoco is Guided Unsupervised Reinforcement Learning. It doesn't use paired data, but it isn't flying totally blind either.

The Cycle-Consistency Engine

The "secret sauce" is a reward term based on cycle-consistency (a concept popularized by CycleGAN in computer vision). The system trains two mapping networks alongside the robot's control policy:

  • Robot2Human-Mapper (R2H): Takes a robot pose and predicts what the equivalent human pose would look like.
  • Human2Robot-Mapper (H2R): Takes that predicted human pose and tries to map it back to the original robot pose.

If the robot's motion is "meaningful" in a human context, the system should be able to cycle from robot to human and back to the exact same robot pose. This maximizes the mutual information between the two domains.

The "Guided" Part

Pure unsupervised learning often results in the robot doing nothing to avoid penalties. To prevent this, the authors add a root-tracking reward. This provides a "hint" to the robot: "Your joints can do whatever, but your main body (the root) should generally follow the path the human is taking." This guidance narrows the search space enough for the cycle-consistency reward to learn fine-grained joint movements.

Datasets

  • Human Motion: The researchers used the LaFAN1 dataset, which contains high-quality human motion capture data. They selected 50 trajectories covering walking, running, hopping, and dancing.
  • Robot Platform: The Unitree Aliengo, a sophisticated quadrupedal robot with 12 joints, simulated within the NVIDIA Isaac Gym physics engine.
  • Application Modules: For extended features, they integrated the Human Motion Diffusion Model (MDM) for text-to-motion and Learned Motion Matching (LMM) for interactive joystick control.

Key Insights and Ideas

  • Reconstructibility is Proxy for Meaning: The "aha moment" here is that a robot doesn't need to look like a human to act like one. As long as the robot's state can be mathematically "reconstructed" back into the human's pose, the semantic meaning of the motion is preserved.
  • Emergent Gaits: The robot doesn't just move its legs randomly; it discovers distinct strategies. For example, when mimicking different human walking styles, the robot naturally switches between "trotting" and "galloping" gaits.
  • Overcoming the "Conservative" Trap: Without the cycle-consistency reward (the "Task-Only" baseline), the robot tends to stand still or move very cautiously. The drive to match human motion encourages the robot to be more "agile" and "creative."

Key Results

The authors compared CrossLoco against DeepMimic (a gold-standard baseline that requires manual engineering) and several ablated versions of their own model.

  • Quantitative Superiority: CrossLoco achieved the highest Averaged Correspondence Reward (ACR) and Diversity (DIV) scores, meaning it stayed truer to the human input while exhibiting a wider range of motions.
  • User Preference: In a study with 15 subjects, users correctly matched the robot's motion to the source human motion 65.5% of the time (beating all but the manually-engineered baseline). More importantly, users preferred CrossLoco's motions (43% preference) over all other methods.
  • Cross-Application Utility: The researchers successfully demonstrated Language2Robot (e.g., typing "strides swiftly in a straight line" makes the robot run) and Interactive Control via a joystick, where the robot adjusted its style in real-time.

Limitations and Future Work

  • High-Frequency Struggles: The system still finds "high-frequency" motions like hopping difficult to translate perfectly, likely due to the extreme differences in how humans and quadrupeds manage ground impact.
  • Morphological Hard Limits: Some human motions (like a very sharp 180-degree turn) are physically impossible for the quadruped to mimic perfectly without falling, leading to some "imperfections" in the translation.
  • Next Steps: The authors aim to move beyond simple locomotion to long-horizon tasks and legged manipulation (e.g., a quadruped with a robotic arm mimicking a human reaching for a tool). They also plan to test the framework on physical hardware in the real world.

Summary updated

Abstract

Human motion driven control (HMDC) is an effective approach for generating natural and compelling robot motions while preserving high-level semantics. However, establishing the correspondence between humans and robots with different body structures is not straightforward due to the mismatches in kinematics and dynamics properties, which causes intrinsic ambiguity to the problem. Many previous algorithms approach this motion retargeting problem with unsupervised learning, which requires the prerequisite skill sets. However, it will be extremely costly to learn all the skills without understanding the given human motions, particularly for high-dimensional robots. In this work, we introduce CrossLoco, a guided unsupervised reinforcement learning framework that simultaneously learns robot skills and their correspondence to human motions. Our key innovation is to introduce a cycle-consistency-based reward term designed to maximize the mutual information between human motions and robot states. We demonstrate that the proposed framework can generate compelling robot motions by translating diverse human motions, such as running, hopping, and dancing. We quantitatively compare our CrossLoco against the manually engineered and unsupervised baseline algorithms along with the ablated versions of our framework and demonstrate that our method translates human motions with better accuracy, diversity, and user preference. We also showcase its utility in other applications, such as synthesizing robot movements from language input and enabling interactive robot control.
BibTeX
@inproceedings{li2024crossloco,
  author = {Tianyu Li and Hyunyoung Jung and Matthew C. Gombolay and Yong Kwon Cho and Sehoon Ha},
  title = {{CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=i0FTEXdPbF},
}
Figure