METATRON: A Promptable Whole-Body Execution Layer for Humanoids

Chen Tessler, Yifeng Jiang, Alec Otto, Yi Shi, Umar Iqbal, Gal Chechik, Xue Bin PengarXiv 2026 · arXiv

TL;DR

METATRON is a "promptable" execution layer for humanoids that separates task intent from physical coordination. By using a single frozen policy that responds to sparse geometric "sketches," it allows new skills—like sitting, walking, or recovering from a fall—to be added via simple code or LLM-authored programs without any further reinforcement learning.

What Problem Does This Paper Address?

Humanoid robots are notoriously difficult to control because they require constant, whole-body coordination just to stay upright. Traditionally, researchers have built "siloed" controllers: one for locomotion, one for manipulation, and another for recovery. This approach is brittle; transitions between these controllers are engineering nightmares, and training a new neural network for every specific task is computationally expensive and doesn't scale.

The authors of METATRON argue for a "separation of concerns." They want to decouple what a motion should accomplish (the intent) from how to coordinate the 20+ joints of a humanoid body (the execution). The goal is a single, general-purpose "motor foundation" that can be steered by simple commands.

Methods

METATRON consists of two main layers: a generative base policy and a programmable skill layer.

  1. The Base Policy: This is the "execution layer." It is trained via a two-stage process. First, "expert" trackers are trained on dense motion-capture data using Reinforcement Learning (PPO). Second, these experts are distilled into a single student policy using a Conditional Variational Autoencoder (CVAE) architecture. Crucially, during distillation, the inputs are randomly masked. This teaches the policy to "fill in the blanks"—it learns to produce a full, coordinated motion even when given only a sparse "prompt" (e.g., "put the left foot here at time $t+0.5$").
  2. The Skill Layer: Once the base policy is trained, it is frozen. To create a new behavior, you don't retrain the network; you write a "skill"—a small piece of code that generates prompts for the policy. These prompts can include timed body-part targets, velocity commands, or "style cues" (biasing the motion to look like a specific clip).
Fig. 1: One frozen controller per embodiment, diverse behaviors. Partial prompts specify goals; each controller provides whole-body coordination across tasks without task- specific training. Teal and gray blocks indicate specified and unspecified goal components, respectively.

A standout feature is the use of LLM coding agents. The authors show that an LLM can analyze motion data, fit geometric rules (like a periodic function for foot placement), and write the skill code itself, effectively "programming" the robot's motor behavior.

Fig. 3: Agent-assisted skill development. Given a task specification and optional motion examples, a human and LLM agent define evaluation scenarios and criteria. The LLM agent then iteratively refines the skill code using simulation feedback. Skills specify goals for the frozen METATRON policy, enabling the model to perform new tasks without additional training.

Datasets

The model is trained on a massive, diverse collection of motion data:

  • BONES: A large-scale skeletal embodiment dataset.
  • AMASS & LAFAN: Standard libraries for human motion capture.
  • SAMP: Used for scene-interaction motions like sitting on furniture or lying on beds.
  • Synthetic Recoveries: Because mocap rarely includes people falling and getting back up in ways a robot can replicate, the authors generated synthetic "get-up" trajectories in simulation to expand the robot's repertoire.

Key Insights and Ideas

  • The "Sketch" Philosophy: The policy is robust enough that it doesn't need a perfect, frame-by-frame reference trajectory. A few geometric points (a "sketch") are sufficient for the policy to handle balance and weight transfer automatically.
  • Frozen Policies for Stability: By freezing the base policy, the authors avoid "catastrophic forgetting." The robot doesn't lose its ability to walk when you teach it to sit; you're simply giving it a new set of instructions for the same motor engine.
  • Style as a Bias: Unlike traditional controllers that enforce a strict pose, METATRON treats "style" as a soft cue. You can tell the robot to "walk toward that point in the style of a crawl," and it will merge those intents naturally.

Key Results

The researchers deployed METATRON on the Unitree G1 humanoid and a simulated character called SOMA.

  • Versatility: A single frozen G1 checkpoint successfully performed walking, teleoperated object pickup, sitting on a sofa, and recovering from diverse fallen positions.
  • Superior Reaching: In whole-body reaching tasks, METATRON achieved an 83% success rate, more than doubling the performance of state-of-the-art reference-tracking pipelines like ARDY+SONIC (39%).
  • Robustness: METATRON proved significantly more stable under physical stress. It could withstand torso pushes of up to 1000N while walking, whereas baseline pipelines failed entirely above 600N.
Fig. 2: Model and training. The prior maps partial prompts and observation history to a deterministic condition and a latent distribution. The decoder combines the condition with a sphere-projected latent to produce joint targets. Training adds a posterior conditioned on the full reference and supervision from a frozen tracker. Only the solid path runs at deployment. Pose renders illustrate state, not camera inputs.

Limitations and Future Work

  • Data Dependency: The robot's repertoire is strictly limited by its training data. If the dataset doesn't contain a specific type of movement (e.g., swimming), no amount of clever prompting will make the robot do it.
  • High-Level Planning: METATRON is an execution layer. It still requires an external system (or a human) to decide where the chair is or when to pick up the apple.
  • Glitches at the Edge: Abruptly commanding high velocities from a standstill can sometimes trigger unintended behaviors, like the robot accidentally performing a front roll because it misinterprets the "intent" as a high-energy athletic move.

Future work aims to integrate real-time perception directly into the prompt generation, allowing for fully autonomous navigation and interaction in unstructured environments.

Summary updated

Abstract

Humanoid controllers often require task-specific training despite sharing the need for balance and whole-body coordination. We present Metatron, a promptable whole-body execution layer combining a generative base policy with a programmable skill layer. The base policy receives partial goals, collectively called a prompt, that can combine timed body-part targets, velocity commands, and style cues. It produces diverse whole-body motions consistent with these goals. Training combines motion capture, synthetic recovery trajectories, and retargeted scene-interaction motions. After training, the policy is frozen. Stateful programs called skills express task intent as prompts at each control step, drawing rough geometric sketches that the policy turns into coordinated movement. Skills are ordinary code, refined by coding agents that analyze motion data and fit compact rules without task-specific policy training. On the physical Unitree G1, one frozen checkpoint supports walking, recovery, teleoperated object pickup, and sitting on furniture. Comparative evaluations show higher reaching success and push robustness than the tested reference-tracking pipelines. The same recipe produces climbing and acrobatics on a simulated character. Metatron enables coding agents to turn learned motor capabilities into new skills through code, without further policy training.
BibTeX
@misc{tessler2026metatron,
  author = {Chen Tessler and Yifeng Jiang and Alec Otto and Yi Shi and Umar Iqbal and Gal Chechik and Xue Bin Peng},
  title = {{METATRON: A Promptable Whole-Body Execution Layer for Humanoids}},
  year = {2026},
  eprint = {2610.38027},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38027},
}
Figure