OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptation

Yiren Song, Xiyao Deng, Pei Yang, Yihan Wang, Mike Zheng ShouarXiv 2026 · arXiv

TL;DR

OmniHumanoid is a novel video generation framework that scales up data production for embodied AI by transferring human or robot motions to new robot embodiments. By decoupling transferable motion dynamics from robot-specific appearances, it allows quick adaptation to unseen robots using only unpaired videos and achieves real-time, streaming 720p video generation via a causal distillation pipeline.

What Problem Does This Paper Address?

Training humanoid robots requires vast amounts of video data showing how robots interact with the physical world. However, physically collecting high-quality demonstrations for every new robot model is prohibitively expensive and slow.

An appealing alternative is cross-embodiment video generation—taking an existing video (e.g., a human performing a task) and "robotizing" it so it looks like a specific target robot executing the exact same action.

Current methods struggle with this because they entangle motion with the robot's unique geometry and kinematics. Furthermore, most existing frameworks require expensive paired data (the exact same motion sequence performed by both the source and target embodiment) for every new robot model, severely limiting scalability. They are also typically too slow for real-time or interactive data generation.

Methods

OmniHumanoid operates under the TAPE principle: Transferable motion, Paired-free adaptation, embodiment Preservation, and generation Efficiency. Built on a Diffusion Transformer (DiT) backbone, the framework splits video generation into two independent pathways:

  1. Shared Motion Transfer Model: This module runs on a conditioning branch, learning embodiment-invariant features like temporal dynamics, object interactions, and scene evolution from a multi-embodiment dataset.
  2. Embodiment Video LoRAs: These are lightweight Low-Rank Adaption modules attached strictly to the denoising branch. Each LoRA is dedicated to a specific robot, capturing its unique geometry, textures, and structural proportions.

To keep these components from interfering with one another, the authors introduce a Unidirectional Information Flow design using an asymmetric attention mask. The denoising branch can read motion cues from the conditioning branch, but the motion branch is completely blinded to the embodiment-specific details.

Figure 2: Overview of OmniHumanoid. A Shared Motion Transfer Model learns transferable motion from motion-aligned paired videos, while each Embodiment Video LoRA captures the appearance and morphology of a specific humanoid embodiment from unpaired videos. During paired training, the target embodiment LoRA is activated on the denoising branch, while the shared motion transfer parameters are optimized on the conditioning branch. This branch-decoupled design enables paired- free adaptation to unseen embodiments while preserving transferable motion learning.

Streaming Video-to-Video Distillation

To make the model practical for large-scale data generation, the authors distilled the computationally heavy bidirectional teacher model into a causal streaming student model. By organizing tokens into an interleaved "super-chunk" layout, the model generates video autoregressively. This distillation compresses the generation process from 50 denoising steps down to just 4 steps.

Datasets

The authors introduced a massive new synthetic dataset and paired it with real-world testing:

  • Human-Humanoid 4D Dataset: Built using Unity HDRP and the Humoto motion library, this dataset features over 700 humanoid motion sequences retargeted across 10 distinct assets (5 robots, 5 human avatars) in 100 diverse scenes. It totals 167 hours and 14.4 million frames of perfectly motion-aligned paired videos.
  • Evaluation Split: To rigorously test paired-free adaptation, one robot embodiment (Unitree-G1) was completely held out from training and used exclusively for testing.
  • Real-world Benchmark: 50 in-the-wild videos of human daily activities and online robot demonstrations were collected to evaluate performance under realistic, unconstrained visual conditions.

Key Insights and Ideas

  • Decoupling Prevents Interference: Directly mixing source-motion conditioning with target-embodiment rendering causes severe visual artifacts. Forcing a unidirectional information flow prevents embodiment priors from polluting the shared motion model.
  • Zero-Shot-Like Scaling via Unpaired Data: Because the shared motion model handles "how things move," integrating an entirely new robot doesn't require paired motion data. Engineers only need to record a few dozen unpaired videos of the new robot to train a lightweight appearance adapter.
  • Autoregressive Video Generation: Treating video tokens like block-wise causal chunks allows video diffusion models to function similarly to large language models, enabling infinite-horizon streaming generation.

Key Results

OmniHumanoid significantly outperforms state-of-the-art commercial and open-source video-to-video baselines (including Kling, Runway Gen-4, and X-Humanoid):

  • Higher Fidelity: On the synthetic benchmark for the held-out Unitree-G1 robot, OmniHumanoid achieved a PSNR of 25.47 and SSIM of 0.9039, comfortably beating X-Humanoid (PSNR 23.03, SSIM 0.8891).
  • Real-Time Speedups: The streaming distillation pipeline boosted generation speeds from a sluggish 0.10 FPS (teacher model) to 4.96 FPS (student model) at 720p resolution on a single NVIDIA H200 GPU.
  • Human Preference: In a user study evaluating motion fidelity and embodiment similarity, OmniHumanoid won 72.7% and 65.7% of user preference votes, respectively, compared to less than 17% for commercial models like Kling O3.

Limitations and Future Work

The primary limitation lies in the speed-quality trade-off of the distillation process. While the 4-step streaming student model is fast enough for real-time applications, it suffers a noticeable drop in fine-grained details, temporal smoothness, and complex motion fidelity when compared directly to the high-quality outputs of the 50-step teacher model.

The authors note that current distillation frameworks struggle to perfectly approximate complex data distributions in so few steps, marking advanced knowledge distillation strategies as a key focus for future work.

Summary updated

Abstract

Cross-embodiment video generation aims to transfer motions across different humanoid embodiments, such as human-to-robot and robot-to-robot, enabling scalable data generation for embodied intelligence. A major challenge in this setting is that motion dynamics are partly transferable across embodiments, whereas appearance and morphology remain embodiment-specific. Existing approaches often entangle these factors, and many require paired data for every target embodiment, which limits scalability to new robots. We present OmniHumanoid, a framework that factorizes transferable motion learning and embodiment-specific adaptation. Our method learns a shared motion transfer model from motion-aligned paired videos spanning multiple embodiments, while adapting to a new embodiment using only unpaired videos through lightweight embodiment-specific adapters. To reduce interference between motion transfer and embodiment adaptation, we further introduce a branch-isolated attention design that separates motion conditioning from embodiment-specific modulation. In addition, we construct a synthetic cross-embodiment dataset with motion-aligned paired videos rendered across diverse humanoid assets, scenes, and viewpoints. Experiments on both synthetic and real-world benchmarks show that OmniHumanoid achieves strong motion fidelity and embodiment consistency, while enabling scalable adaptation to unseen humanoid embodiments without retraining the shared motion model.
BibTeX
@misc{song2026omnihumanoid,
  author = {Yiren Song and Xiyao Deng and Pei Yang and Yihan Wang and Mike Zheng Shou},
  title = {{OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptation}},
  year = {2026},
  eprint = {2605.12038},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.12038},
}
Figure