In the quest for a "Generalist Robot Policy"—a single AI that can control any robot to do anything—researchers face a fundamental hurdle: embodiment gap. A joint angle of 30 degrees means something completely different on a long-armed industrial robot versus a compact desktop manipulator. If we want one model to rule them all, it needs to understand the "spatial meaning" of its actions across diverse hardware.
RoboMorpher, a new framework from researchers at HKU, HKUST, and other institutions, tackles this by introducing a morphology-conditioned action tokenizer that grounds control signals in physical reality.
TL;DR
RoboMorpher is a framework that learns shared action representations across different robot designs by pretraining an action tokenizer on 40 million procedurally generated kinematic pairs. By forcing the model to reconstruct both joint angles and physical link poses, it achieves significantly better cross-embodiment performance on complex manipulation tasks compared to standard joint-space control.
What Problem Does This Paper Address?
Most current robot learning methods represent actions as a vector of joint values. However, identical joint values produce different spatial configurations depending on link lengths and joint arrangements. This ambiguity makes it incredibly difficult for a model trained on a Franka Panda to generalize to a UR5 or a custom-built robot.
While End-Effector (EE) control (specifying Cartesian targets) avoids some of this, it requires robot-specific inverse kinematics and doesn't uniquely define the entire arm's pose. The authors argue that a truly shared representation must preserve control coordinates while accounting for their morphology-dependent spatial effects.
Methods
The researchers introduced three key components to bridge this gap:
- MorphGen: A procedural generator that creates millions of synthetic robot arm descriptions (URDFs). It uses "survey-derived geometric priors"—statistical distributions of link lengths and joint types from 33 real-world robot descriptions—to ensure the generated robots are physically plausible.
- RoboMorpher Tokenizer: This is the heart of the system. It's an encoder-decoder architecture that takes a robot's morphology and its joint angles as input. It converts these into discrete "action codes." Crucially, during pretraining, the decoder must reconstruct not just the joint angles but also the 3D positions and rotations of every link (link poses). This forces the codes to capture the kinematic relationship between control and spatial effect.
- VLA Policy Integration: The frozen RoboMorpher tokenizer is plugged into a Vision-Language-Action (VLA) policy based on the Qwen3-VL model. It uses a dual-branch Diffusion Transformer (DiT): one branch predicts discrete action codes (coarse trajectory), and another refines them into continuous actions using flow-matching.
Datasets
- Pretraining Data: 40 million morphology–state pairs generated by MorphGen.
- RoboTwin 2.0: A benchmark featuring 15 tasks across 5 distinct robot embodiments.
- EmbodiBench (New): A challenging new benchmark introduced in this paper. it consists of 10 tabletop tasks (like picking, lifting, and stacking) evaluated across 30 different generated robot morphologies per task (300 unique robot-task combinations in total).
Key Insights and Ideas
- Kinematic Supervision is the "Secret Sauce": Simply learning to predict joint angles isn't enough for cross-embodiment. By adding a loss for reconstructing link poses (Forward Kinematics), the model learns the "geometry" of the action.
- Scaling via Procedural Generation: You don't need millions of real-world demonstrations to learn robot kinematics. MorphGen shows that survey-guided synthetic data is a scalable and effective substitute for teaching a model how different bodies move.
- Morphology-Conditioned Tokenization: By using AdaLN-Zero and graph-biased attention, the tokenizer can dynamically adjust its internal representation based on the specific robot it is currently "inhabiting."
Key Results
- Superior Performance: RoboMorpher achieved a 56.40% success rate on RoboTwin 2.0 and 23.27% on EmbodiBench, outperforming standard continuous joint-space control by 9.73 and 15.34 percentage points, respectively.
- The Importance of Link Poses: In ablation studies, removing the link-pose reconstruction supervision caused success on EmbodiBench to plummet from 23.27% to just 5.33%. This confirms that spatial grounding is vital for generalizing to new robots.
- Generalization to Unseen Robots: The model was tested on robot designs (URDFs) that were never seen during the tokenizer's pretraining, demonstrating robust zero-shot kinematic reconstruction.
Limitations and Future Work
- No Zero-Shot Policy Transfer: While the tokenizer generalizes to new robots, the VLA policy still requires some task-specific demonstrations for the target embodiments to achieve high success.
- Reconstruction Outliers: While average reconstruction error is low, the authors noted substantial outliers in orientation and position for some generated URDFs, suggesting the need for even more robust training.
- Difficult Tasks: Complex tasks like high-precision stacking remain a significant challenge for all evaluated models, with success rates near zero.
- Limited Survey Scope: The current geometric priors are based on a survey of 33 robots; expanding this to include a wider variety of specialized manipulators could further improve the diversity of MorphGen.
Summary updated