Humanoid World Models: Open World Foundation Models for Humanoid Robotics

Muhammad Qasim Ali, Aditya Sridhar, Shahbuland Matiana, Andy Wong, Mohammad K. Al-SharmanarXiv 2025 · DOI · arXiv

Predictive Intelligence for Humanoid Robotics

Humanoid robots are designed to operate in complex, human-centric environments. Navigating these "open worlds" requires more than just reactive control; it demands predictive intelligence. World models serve as the "imagination" of a robot, allowing it to predict the future consequences of its actions before executing them. This capability is essential for long-horizon planning and generating synthetic data to train robust control policies.

However, existing world models often fall into two extremes: either they are massive, closed-source systems requiring industrial-scale compute, or they are designed for entertainment (e.g., video generation) without the physical grounding necessary for robotics. Researchers have introduced Humanoid World Models (HWM) to bridge this gap, offering a family of lightweight, open-source foundation models specifically tailored for humanoid morphology and egocentric perception.

Conceptual overview of the Humanoid World Model predicting future video from past frames and action tokens.

Architectural Innovations: Efficiency through Parameter Sharing

The HWM project explores two primary generative paradigms: Masked Transformers (Masked-HWM) and Flow-Matching (Flow-HWM). While Masked Transformers treat video latents as discrete tokens to "fill in" future frames, Flow-Matching models learn a vector field to transform noise into structured video.

A central contribution of this work is the exploration of parameter-sharing techniques to reduce model size without sacrificing performance. The researchers tested several transformer block designs, focusing on:

  • Joint vs. Split Attention: Determining whether video and action tokens should be processed simultaneously or in distinct stages.
  • Weight Sharing: By implementing "Full Sharing" (across all data streams) or "Modality Sharing" (within specific data types), the models achieved a size reduction of 33–53%. This hybrid approach allows early layers to capture modality-specific features while deeper layers handle complex cross-modal reasoning.
Architecture of the Masked-HWM, utilizing spatial and temporal attention mechanisms for video prediction.

Training on Real-World Humanoid Data

The models were trained using the 1xGPT dataset from 1X Technologies, which consists of 100 hours of egocentric video from the Humanoid EVE Android. The data includes 30Hz video paired with 25-dimensional action vectors representing joint angles and velocities (wrist, knee, elbow, shoulder, neck, and hip).

By designing the models to run on just 1–2 consumer-grade GPUs (e.g., NVIDIA A6000), the authors have made foundation model research accessible to academic labs and smaller research teams, breaking the "Big Tech" compute barrier.

Performance and Visual Fidelity

The evaluation revealed several key insights:

  • Masked Transformers Lead: In the specific context of humanoid robotics and limited compute, Masked-HWM consistently outperformed Flow-HWM in both visual quality (achieving a top FID score of 10.13) and inference speed.
  • Joint Attention: For Masked-HWM, allowing all tokens to interact via joint temporal attention proved most effective.
  • Efficiency: The "Full Sharing" variant of Masked-HWM compressed the model to just 0.195 Billion parameters, maintaining high fidelity while reaching inference speeds of 2.36 samples per second.

Qualitatively, the models successfully render complex scene structures, such as furniture and textures, and accurately depict the robot's own appendages across varying lighting conditions.

Limitations and the Path Forward

While HWM represents a significant step toward accessible humanoid foundation models, challenges remain. Fine-grained details, such as individual finger movements, can appear blurry or entangled. Additionally, "temporal drift" can occur in longer sequences, where geometric shapes lose their definition over time. The models also struggle with out-of-distribution (OOD) actions, often defaulting to a "canonical" arm appearance when faced with unfamiliar commands.

Future research will focus on scaling the training datasets and integrating these world models into closed-loop control systems, enabling robots to use their "imagination" for real-time error correction and path planning.

Summary updated

BibTeX
@misc{ali2025humanoid,
  author = {Muhammad Qasim Ali and Aditya Sridhar and Shahbuland Matiana and Andy Wong and Mohammad K. Al-Sharman},
  title = {{Humanoid World Models: Open World Foundation Models for Humanoid Robotics}},
  year = {2025},
  eprint = {2506.01182},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.01182},
}
Figure