MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation

Yutong Shen, Hangxu Liu, Penghui Liu, Jiashuo Luo, Yongkang Zhang, Rex Morvley, Chen Jiang, Jianwei Zhang, Lei Zhang

TL;DR

MetaWorld-X is a hierarchical framework that masters complex humanoid "loco-manipulation" (simultaneously moving and doing) by orchestrating a library of specialized experts. By using human motion priors for natural movement and a Vision-Language Model (VLM) to semantically route these experts, it achieves significantly higher success rates and more lifelike motions than traditional monolithic reinforcement learning methods.

What Problem Does This Paper Address?

Controlling a high-degree-of-freedom (DoF) humanoid robot is a nightmare for standard Reinforcement Learning (RL). Most current approaches use a "monolithic" policy—one giant neural network trying to learn everything at once. This leads to gradient interference, where the weights needed to balance while walking conflict with the weights needed to precisely reach for a door handle.

The result is often "robotic" in the worst sense: jittery movements, unnatural gaits, and a total lack of stability when tasks get complex. Furthermore, these models struggle to generalize; if a robot learns to "walk" and "reach" separately, it often fails to combine them into "walk to the door and reach for the handle" without extensive retraining.

Methods

MetaWorld-X tackles this using a "divide-and-conquer" hierarchical architecture composed of three main pillars:

  1. Specialized Expert Policies (SEP): Instead of one policy, the authors train a pool of experts for fundamental primitives (Stand, Walk, Run, Sit, Carry, Reach, Crawl).
  2. Imitation-Constrained RL: To ensure the robot doesn't move like a glitching ragdoll, experts are trained using human motion priors. They use an energy-based alignment reward that forces the robot to stay biomechanically consistent with human data while still optimizing for the task goal.
  3. Intelligent Routing Mechanism (IRM): This is the "brain" that orchestrates the experts. It uses a Vision-Language Model (VLM), such as GPT-4o, to provide high-level semantic guidance. The VLM looks at the task (e.g., "Open the door") and the visual state to determine the "expert weights"—essentially deciding how much of the "Walk" expert vs. the "Reach" expert should be active at any given millisecond.

The system is built on top of a World Model (similar to TD-MPC2), which allows the robot to "imagine" outcomes in a latent space, making training much more sample-efficient.

Datasets

  • AMASS: A large-scale archive of human motion capture data used to provide the "human motion priors" for the experts.
  • Humanoid-bench: The primary evaluation benchmark. It includes locomotion tasks (walking, running) and complex loco-manipulation tasks like "Basketball," "Trucking" (carrying a box to a truck), and "Door opening."

Key Insights and Ideas

  • Semantic Routing > Heuristic Routing: Traditional Mixture-of-Experts (MoE) models often struggle to learn which expert to use when. By leveraging the "common sense" of a VLM, MetaWorld-X knows that "opening a door" requires a transition from walking to reaching without needing to fail thousands of times to figure that sequence out.
  • Decoupling Primitives: By separating the experts, the authors eliminate gradient interference. The "Walk" expert can focus entirely on a stable gait without its weights being pulled apart by the requirements of a "Reach" task.
  • Energy-Based Alignment: The authors found that treating imitation as an energy minimization problem (rather than just copying joint angles) makes the robot much more robust to "retargeting" errors—the inherent physical differences between a human skeleton and a robot model.

Key Results

  • Superior Success Rates: On basic locomotion, MetaWorld-X achieved a 90% success rate, while the strong baseline TD-MPC2 hovered around 30-40%.
  • Efficiency: The model converged significantly faster. For the "Walk" task, MetaWorld-X reached peak performance in 0.5 million steps, compared to 1.8 million or more for baselines.
  • Zero-Shot Generalization: Thanks to the VLM router, the system could handle multi-stage tasks (like walking to a box, picking it up, and carrying it) with "few-shot" behavioral refinement, whereas monolithic models often failed to chain these behaviors together.
  • Motion Quality: Qualitatively, the motions are much smoother. The paper highlights that baselines often exhibit "unnatural leg lifting" or "excessive arm swing," while MetaWorld-X produces gaits that closely resemble biological locomotion.

Limitations and Future Work

While MetaWorld-X is a leap forward, the authors note a few areas for growth:

  • Skill Library Scaling: The current model uses eight core experts. Scaling this to dozens or hundreds of skills will require even more sophisticated routing.
  • Real-World Gap: The experiments were conducted in the MuJoCo simulator. Moving from simulation to a physical humanoid will introduce sensor noise and hardware constraints not fully captured here.
  • Tightening the Loop: Future work will look at more deeply integrating the VLM's semantic reasoning directly into the world model's predictive dynamics, rather than just using it as a high-level supervisor.

Summary updated

Abstract

Learning natural, stable, and compositionally generalizable whole-body control policies for humanoid robots performing simultaneous locomotion and manipulation (loco-manipulation) remains a fundamental challenge in robotics. Existing reinforcement learning approaches typically rely on a single monolithic policy to acquire multiple skills, which often leads to cross-skill gradient interference and motion pattern conflicts in high-degree-of-freedom systems. As a result, generated behaviors frequently exhibit unnatural movements, limited stability, and poor generalization to complex task compositions. To address these limitations, we propose MetaWorld-X, a hierarchical world model framework for humanoid control. Guided by a divide-and-conquer principle, our method decomposes complex control problems into a set of specialized expert policies (Specialized Expert Policies, SEP). Each expert is trained under human motion priors through imitation-constrained reinforcement learning, introducing biomechanically consistent inductive biases that ensure natural and physically plausible motion generation. Building upon this foundation, we further develop an Intelligent Routing Mechanism (IRM) supervised by a Vision-Language Model (VLM), enabling semantic-driven expert composition. The VLM-guided router dynamically integrates expert policies according to high-level task semantics, facilitating compositional generalization and adaptive execution in multi-stage loco-manipulation tasks.
BibTeX
@misc{shen2026metaworld,
  author = {Yutong Shen and Hangxu Liu and Penghui Liu and Jiashuo Luo and Yongkang Zhang and Rex Morvley and Chen Jiang and Jianwei Zhang and Lei Zhang},
  title = {{MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation}},
  year = {2026},
  eprint = {2603.08572},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.08572},
}
Figure