A Unified Inertial Encoder for Human and Humanoid Motion

Yutian Chen, Shibo Zhao, Rocky Duan, Karen Liu, Carmelo Sferrazza, Guanya Shi, Yuheng Qiu, Koushil SreenatharXiv 2026 · arXiv

Humanoid robots are finally stepping out of the lab and into the real world, but they still struggle with two fundamental tasks: knowing exactly how they are moving (state estimation) and understanding what they are doing in plain English (language-grounded motion understanding). Traditionally, these capabilities are trained on data recorded from the robots themselves. This creates a "chicken-and-egg" problem: a robot can only learn from the motions its current controller can execute, which limits its ability to handle new, complex behaviors.

In this paper, researchers from Amazon FAR, UC Berkeley, Stanford, and CMU propose a way to break this cycle by allowing humanoid robots to learn directly from the vast and diverse datasets of human motion.

TL;DR

The authors developed a Unified Inertial Encoder that enables humanoid robots to learn state estimation and motion understanding from human data. By using a shared anatomical interface and "kinematic IMU synthesis," their model achieves state-of-the-art zero-shot performance on a real Unitree G1 robot without ever seeing a single frame of real-robot training data.

What Problem Does This Paper Address?

The core challenge in transferring human motion knowledge to robots is embodiment mismatch. Humans and robots have different body geometries, different sensor placements, and different physical noise profiles (like the vibrations of a robot's motors vs. the soft tissue of a human).

Existing methods usually specialize in one platform. If you want a robot to recognize "climbing stairs," you typically need to record a robot climbing stairs. But human datasets already have thousands of hours of diverse motions—everything from dancing to limping—with rich text descriptions. The goal here is to tap into that "human prior" to make robots smarter and more robust.

Methods

The researchers introduced three key innovations to bridge the human-robot gap:

  1. Shared Inertial Interface & Kinematic Synthesis: They defined 12 anatomical sites (like the head, pelvis, and limbs) common to both humans and humanoids. While humans in datasets often wear 12 physical IMUs, most robots only have one (usually in the pelvis). To match the human data, the authors used Kinematic IMU Synthesis. By combining the robot's single pelvis IMU with its joint encoder data and its physical model (URDF), they "transport" the inertial signals to the other 11 sites, creating a virtual 12-IMU setup that looks just like a human's.
The shared inertial interface maps human IMU sites to synthesized robot sites.
  1. Continuous-Time Encoding: Different datasets use different sampling rates (e.g., 60Hz vs. 1000Hz). Instead of messy resampling that can introduce artifacts, they used hierarchical continuous convolutions. This allows the model to process sensor data at its native frequency, handling missing samples or irregular timing gracefully.
  2. Multi-Task Joint Training: The model is trained simultaneously on two tasks:
  3. Velocity Estimation: Predicting how fast each body part is moving to enable inertial odometry.
  4. Motion-Text Alignment: Aligning the IMU features with a frozen CLIP text encoder so the robot can "understand" actions like "walking" or "standing" through language.
The unified encoder architecture processes diverse human and humanoid data.

Datasets

The model was trained on a massive combined corpus:

  • Human Data: Nymeria (300+ hours), Natural Motion (40.6 hours), and BONES-Seed (144 hours) of real human motion with IMU and language labels.
  • Humanoid Data: BONES-Seed G1 (144 hours), which consists of human motions kinematically retargeted to the Unitree G1 robot model.
  • Evaluation: A separate Real-Trajectory G1 dataset of physical robot recordings was used for zero-shot testing.

Key Insights and Ideas

  • Human data is a powerful prior: Training on human data doesn't just add more samples; it provides a "common sense" of motion that helps the robot generalize to movements it hasn't seen in a robotic context.
  • Virtual sensors are enough: You don't need to bolt 12 IMUs onto a robot. Kinematic synthesis is accurate enough to serve as a bridge for cross-embodiment learning.
  • Sim-to-Real via Randomization: To handle the gap between "clean" synthesized signals and "noisy" real-world robot sensors, the authors heavily randomized gravity, sensor noise, bias drift, and even mechanical impacts during training.

Key Results

The model was deployed zero-shot on a physical Unitree G1 robot—meaning it was never trained on real G1 sensor data or in a physics simulator.

  • Superior Odometry: It achieved an absolute velocity error of just 0.099 m/s on the real G1, the lowest among all evaluated methods.
  • Semantic Transfer: The robot could recognize motion categories (like "jogging") that were only present in the human training data, reaching a 53.8% recall@1 for motion-description retrieval.
  • Real-Time Performance: On an NVIDIA L40S, the encoder processes a 4-second window in just 1.78 ms, making it more than fast enough for closed-loop control.
Zero-shot deployment results showing accurate odometry and language retrieval on a real G1.

Limitations and Future Work

While the velocity estimation is highly accurate, the authors note that orientation drift remains a challenge for long-term trajectory tracking. Integrating these learned velocities with traditional filters (like an EKF) for more stable long-term odometry is a key next step. Additionally, the current work focuses on a single robot platform (G1); future work will look at scaling this to a wider variety of humanoid shapes and sizes.

Summary updated

Abstract

Learning humanoid state estimation and language-grounded motion understanding from robot recordings ties training coverage to the robot's motion repertoire. Human motion datasets offer broader coverage, yet transferring this knowledge requires overcoming differences in body geometry, sensing, and physical disturbances. To enable this transfer, we propose a unified inertial encoder whose shared inertial interface uses kinematic IMU synthesis to represent human and humanoid motion at corresponding body sites. Within this common input space, joint training on human and kinematically retargeted humanoid motion combines velocity estimation and motion--text alignment with continuous-time encoding and sensor and motion randomization. The resulting model transfers zero-shot to a physical Unitree G1 without physics simulation, real-robot training trajectories, or fine-tuning, achieving the lowest velocity error among evaluated methods across five datasets, including 0.099m/s on real G1. Its motion-description retrieval also extends to retargeted G1 motion categories observed only in human training, demonstrating how human motion knowledge can support the generalization of humanoid inertial odometry and language-grounded motion understanding.
BibTeX
@misc{chen2026unified,
  author = {Yutian Chen and Shibo Zhao and Rocky Duan and Karen Liu and Carmelo Sferrazza and Guanya Shi and Yuheng Qiu and Koushil Sreenath},
  title = {{A Unified Inertial Encoder for Human and Humanoid Motion}},
  year = {2026},
  eprint = {2611.10137},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10137},
}
Figure