MAWAM: A Multi-Agent World-Action Model for Coordinated Manipulation

Xianzhe Fan, Chi-Pin Huang, Fu-En Yang, Jing Wang, Yecheng Wu, Yu-Cheng Chou, Johan Bjorck, Linxi "Jim" Fan, Jan Kautz, Yu-Chiang Frank Wang, Zhiding Yu, Hengshuang Zhao, Zhiqi LiarXiv 2026 · arXiv

In the world of robotics, coordination is a high-stakes game of "predicting the unpredictable." When multiple robots work together—say, folding a towel or handing over an umbrella—every move one robot makes fundamentally changes the environment for its teammates. To collaborate effectively, robots need more than just a plan; they need physical foresight.

Researchers from NVIDIA, HKU, and MIT have introduced MAWAM (Multi-Agent World-Action Model), a unified generative model that allows a team of robots to "dream" their shared future together. By jointly predicting future camera views and executable actions in a single step, MAWAM enables a level of synchronization that traditional decentralized policies struggle to achieve.

An overview of MAWAM showing real-robot tasks and the joint video-action denoising process.

TL;DR

MAWAM is a multi-agent world-action model that jointly generates future camera views and executable action chunks for an entire robot team in a single generative sample. By treating the team's shared physical future as a coordination space, it achieves state-of-the-art performance on complex collaborative tasks in both simulation and the real world.

What Problem Does This Paper Address?

Coordinating multiple robots is notoriously difficult. Existing approaches usually fall into one of three buckets:

  1. Decentralized policies: Each robot has its own brain. This scales well but often leads to "oscillatory behavior" or awkward waiting because the robots aren't truly in sync.
  2. Joint trajectory generation: These models plan paths for everyone at once but don't "see" the consequences. They lack explicit visual predictions of how the world will change.
  3. Language/Logic coordination: High-level planners that tell robots what to do but ignore the low-level physical nuances of the interaction.

The gap lies in joint world-action modeling. A single robot can use a "world model" to predict what its camera will see after it moves. MAWAM extends this to a team: if Agent A lifts a bag, Agent B needs to see that the bag is now open in its own private camera view to know exactly where to place a garment.

Methods

MAWAM is built upon GR00T 2, a pretrained single-robot world-action backbone. The researchers extended this architecture to handle multiple agents through several key innovations:

  • Joint Video-Action Denoising: Using a technique called flow matching, the model jointly denoises tokens representing future video frames and action chunks for all robots simultaneously.
  • Cross-Agent Attention: A shared Transformer connects all agents. This allows Agent A’s predicted actions to directly influence the predicted future views (and actions) of Agent B.
  • Agent-Attributed Representation: To keep track of who is who, the model uses "regular-simplex rotary identities"—a geometric way to give each robot a unique ID in the attention space without imposing an arbitrary order. It also uses role embeddings to distinguish a robot's identity from its current task (e.g., "giver" vs. "receiver").
  • Two-Stage Training:
    1. Warm-up: The model is first adapted to the specific cameras and controllers of the target robots using single-robot data.
  • Team Training: The model learns from synchronized multi-robot demonstrations, enabling cross-agent attention to capture the "dance" of collaboration.
The MAWAM architecture for joint video-action denoising and training stages.

Datasets

The authors evaluated MAWAM across both physical and simulated environments:

  • Real-Robot Tasks: Three challenging collaborative tasks using Galaxea R1 Pro robots: laundry-bag collection (one holds the bag, the other loads), bath-towel folding, and umbrella handover.
  • Simulation (RoboFactory): Eight tasks involving teams of two to four arms, including Camera Alignment (where one arm holds a camera and another holds the target) and Long Pipeline Delivery (a four-arm relay).

Key Insights and Ideas

  • The Future is the Coordination Space: The most striking insight is that predicting the visual future of the whole team serves as the "shared brain" for coordination. When the model predicts the team's future views, it implicitly computes the dependencies between their actions.
  • Attention is Non-Negotiable: Ablation studies showed that without cross-agent attention, success rates in tightly coupled tasks like the four-arm relay plummeted from 95% to 39%.
  • Identity vs. Role: By separating identity (which robot is it?) from role (what is it doing?), the model can handle teams where multiple robots share the same responsibility (e.g., two robots acting as "camera holders") without getting confused.
  • Robustness to Delays: MAWAM is surprisingly resilient. In experiments where a teammate was intentionally delayed, the MAWAM-controlled receiver naturally waited for its partner to finish its move before proceeding, demonstrating true reactive coordination.

Key Results

MAWAM consistently outperformed the baseline MA-VLA (a multi-agent version of Vision-Language-Action models):

  • Real World: Achieved a 67.8% mean success rate, an 18.9 percentage point improvement over MA-VLA.
  • Simulation: Reached 90.3% mean success across eight RoboFactory tasks, exceeding previous benchmarks by nearly 7 points.
  • Scaling: The model successfully coordinated up to four arms in simulation, reaching 97% success on a complex "Take Photo" task.

Limitations and Future Work

While impressive, MAWAM has a few constraints:

  • Fixed Team Sizes: The current experiments are limited to fixed teams of up to four agents.
  • Inference Latency: Generating joint samples for a whole team is computationally heavier than single-robot inference, though the authors noted that doubling the team size only increased inference time by about 30%.
  • Future Directions: The authors suggest extending MAWAM to handle dynamic team membership (robots joining/leaving mid-task), diverse robot embodiments (different types of robots working together), and even human-robot collaboration, where the model would predict human actions to better coordinate with them.

Summary updated

Abstract

Every move in a robot team changes the world in which its teammates must act. Coordinated manipulation therefore calls for predicting the team's actions together with the physical future they jointly create. We introduce MAWAM, a multi-agent world-action model that jointly generates each robot's future views and executable action chunks in a single sample. Built from a pretrained single-robot world-action backbone, MAWAM jointly denoises all robots' action and future-view tokens through cross-agent attention, while preserving a separate control interface for each robot. MAWAM achieves the highest success on all three real-robot tasks (laundry-bag collection, bath-towel folding, and umbrella handover), with 67.8% mean success, 18.9 percentage points above MA-VLA trained on the same demonstrations and cameras. In simulation, MAWAM reaches 90.3% mean success on eight tasks with teams of up to four arms. Ablations show that cross-agent attention and team-stage future-video supervision both improve coordinated control: without attention, a four-arm relay falls from 95.0% to 39.0%, and without this supervision, Place Food falls from 78.0% to 47.0%.
BibTeX
@misc{fan2026mawam,
  author = {Xianzhe Fan and Chi-Pin Huang and Fu-En Yang and Jing Wang and Yecheng Wu and Yu-Cheng Chou and Johan Bjorck and Linxi "Jim" Fan and Jan Kautz and Yu-Chiang Frank Wang and Zhiding Yu and Hengshuang Zhao and Zhiqi Li},
  title = {{MAWAM: A Multi-Agent World-Action Model for Coordinated Manipulation}},
  year = {2026},
  eprint = {2611.25105},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25105},
}
Figure