RoboMorpher: Scaling Robot Morphology Representations for Cross-Embodiment Action Prediction

Wenwei Lin, Yan Qin, Weiliang Deng, Zixuan Li, Kaixuan Wang, Zanxin Chen, Yue Chen, Tianshuo Yang, Junting Chen, Zhixuan Liang, Yao Mu, Ximin Lyu, Renjing Xu, Ping Luo, Tianxing ChenarXiv 2026 · arXiv

In the quest for a "Generalist Robot Policy"—a single AI that can control any robot to do anything—researchers face a fundamental hurdle: embodiment gap. A joint angle of 30 degrees means something completely different on a long-armed industrial robot versus a compact desktop manipulator. If we want one model to rule them all, it needs to understand the "spatial meaning" of its actions across diverse hardware.

RoboMorpher, a new framework from researchers at HKU, HKUST, and other institutions, tackles this by introducing a morphology-conditioned action tokenizer that grounds control signals in physical reality.

TL;DR

RoboMorpher is a framework that learns shared action representations across different robot designs by pretraining an action tokenizer on 40 million procedurally generated kinematic pairs. By forcing the model to reconstruct both joint angles and physical link poses, it achieves significantly better cross-embodiment performance on complex manipulation tasks compared to standard joint-space control.

What Problem Does This Paper Address?

Most current robot learning methods represent actions as a vector of joint values. However, identical joint values produce different spatial configurations depending on link lengths and joint arrangements. This ambiguity makes it incredibly difficult for a model trained on a Franka Panda to generalize to a UR5 or a custom-built robot.

While End-Effector (EE) control (specifying Cartesian targets) avoids some of this, it requires robot-specific inverse kinematics and doesn't uniquely define the entire arm's pose. The authors argue that a truly shared representation must preserve control coordinates while accounting for their morphology-dependent spatial effects.

Methods

The researchers introduced three key components to bridge this gap:

  1. MorphGen: A procedural generator that creates millions of synthetic robot arm descriptions (URDFs). It uses "survey-derived geometric priors"—statistical distributions of link lengths and joint types from 33 real-world robot descriptions—to ensure the generated robots are physically plausible.
  2. RoboMorpher Tokenizer: This is the heart of the system. It's an encoder-decoder architecture that takes a robot's morphology and its joint angles as input. It converts these into discrete "action codes." Crucially, during pretraining, the decoder must reconstruct not just the joint angles but also the 3D positions and rotations of every link (link poses). This forces the codes to capture the kinematic relationship between control and spatial effect.
  3. VLA Policy Integration: The frozen RoboMorpher tokenizer is plugged into a Vision-Language-Action (VLA) policy based on the Qwen3-VL model. It uses a dual-branch Diffusion Transformer (DiT): one branch predicts discrete action codes (coarse trajectory), and another refines them into continuous actions using flow-matching.
Figure 2. RoboMorpher pipeline. (a) MorphGen samples local-Z chains using survey priors and IK filtering. (b) URDF-conditioned tokenization learns two codes per joint with FK supervision. (c) The VLA predicts codes, decodes ten nodes, interpolates 50 frames, and refines them with flow matching. Frozen URDF features condition both DiTs and the decoder. VLM context conditions the DiTs.

Datasets

  • Pretraining Data: 40 million morphology–state pairs generated by MorphGen.
  • RoboTwin 2.0: A benchmark featuring 15 tasks across 5 distinct robot embodiments.
  • EmbodiBench (New): A challenging new benchmark introduced in this paper. it consists of 10 tabletop tasks (like picking, lifting, and stacking) evaluated across 30 different generated robot morphologies per task (300 unique robot-task combinations in total).
The ten tasks in the new EmbodiBench benchmark, testing generalization across diverse robot designs.

Key Insights and Ideas

  • Kinematic Supervision is the "Secret Sauce": Simply learning to predict joint angles isn't enough for cross-embodiment. By adding a loss for reconstructing link poses (Forward Kinematics), the model learns the "geometry" of the action.
  • Scaling via Procedural Generation: You don't need millions of real-world demonstrations to learn robot kinematics. MorphGen shows that survey-guided synthetic data is a scalable and effective substitute for teaching a model how different bodies move.
  • Morphology-Conditioned Tokenization: By using AdaLN-Zero and graph-biased attention, the tokenizer can dynamically adjust its internal representation based on the specific robot it is currently "inhabiting."
MorphGen uses priors from real robots to sample realistic link lengths and arrangements.

Key Results

  • Superior Performance: RoboMorpher achieved a 56.40% success rate on RoboTwin 2.0 and 23.27% on EmbodiBench, outperforming standard continuous joint-space control by 9.73 and 15.34 percentage points, respectively.
  • The Importance of Link Poses: In ablation studies, removing the link-pose reconstruction supervision caused success on EmbodiBench to plummet from 23.27% to just 5.33%. This confirms that spatial grounding is vital for generalizing to new robots.
  • Generalization to Unseen Robots: The model was tested on robot designs (URDFs) that were never seen during the tokenizer's pretraining, demonstrating robust zero-shot kinematic reconstruction.

Limitations and Future Work

  • No Zero-Shot Policy Transfer: While the tokenizer generalizes to new robots, the VLA policy still requires some task-specific demonstrations for the target embodiments to achieve high success.
  • Reconstruction Outliers: While average reconstruction error is low, the authors noted substantial outliers in orientation and position for some generated URDFs, suggesting the need for even more robust training.
  • Difficult Tasks: Complex tasks like high-precision stacking remain a significant challenge for all evaluated models, with success rates near zero.
  • Limited Survey Scope: The current geometric priors are based on a survey of 33 robots; expanding this to include a wider variety of specialized manipulators could further improve the diversity of MorphGen.

Summary updated

Abstract

Learning shared action representations is central to cross-embodiment manipulation. However, identical joint values can produce different spatial configurations across robots, limiting representations learned from joint values alone. We introduce RoboMorpher, a morphology-conditioned action tokenizer that jointly reconstructs joint angles and link poses to associate control coordinates with their kinematic effects. MorphGen supplies scalable kinematic supervision by generating serial-arm descriptions from survey-derived geometric priors. We pretrain RoboMorpher on 40 million morphology-state pairs and integrate the frozen tokenizer into a vision-language-action policy for discrete prediction and continuous refinement. We also introduce EmbodiBench, a ten-task benchmark with 30 generated URDFs per task. With downstream URDFs excluded from tokenizer pretraining, the policy achieves 56.40% success on RoboTwin 2.0 and 23.27% on EmbodiBench, exceeding continuous joint-space control by 9.73 and 15.34 percentage points. Removing link-pose supervision reduces EmbodiBench success by 17.94 percentage points, supporting morphology-aware geometric supervision for joint-action representation learning.
BibTeX
@misc{lin2026robomorpher,
  author = {Wenwei Lin and Yan Qin and Weiliang Deng and Zixuan Li and Kaixuan Wang and Zanxin Chen and Yue Chen and Tianshuo Yang and Junting Chen and Zhixuan Liang and Yao Mu and Ximin Lyu and Renjing Xu and Ping Luo and Tianxing Chen},
  title = {{RoboMorpher: Scaling Robot Morphology Representations for Cross-Embodiment Action Prediction}},
  year = {2026},
  eprint = {2611.24816},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24816},
}
Figure