TL;DR
The authors introduce URMA (Unified Robot Morphology Architecture), a single, morphology-agnostic neural network policy capable of controlling 16 completely different legged robots—ranging from quadrupeds and hexapods to bipeds and humanoids. By utilizing attention-based encoders and decoders conditioned on physical joint descriptions, URMA learns a generalized locomotion controller that can transfer zero-shot to entirely unseen robots in both simulation and the real world.
What Problem Does This Paper Address?
In deep reinforcement learning (DRL) for legged locomotion, policies are almost always locked to a specific robot. A controller trained for a four-legged Unitree A1 cannot be deployed on an ANYmal, let alone a two-legged humanoid, because their observation and action spaces differ based on the number of joints and limbs.
Traditional Multi-Task Reinforcement Learning (MTRL) attempts to solve this by either padding observation vectors with zeros to match a maximum size or using separate neural network "heads" for each robot. However, these approaches scale poorly, require retraining from scratch for any new robot configuration, and fail to generalize to unseen morphologies. URMA tackles this fundamental "correspondence problem" by creating a single framework that treats different robot bodies as variations of a single underlying control task.
Methods
URMA achieves morphology independence through a clever separation of general state observations and robot-specific joint/foot characteristics:
- General Observations: Fixed-size inputs common to all robots, such as angular velocity from an IMU, command velocities, and gravity vectors.
- Joint Descriptions & Observations: For every joint, the network receives its current state (position, velocity) alongside a fixed-size description vector containing its physical properties (3D relative position, rotation axis, torque/velocity limits, rotor inertia, etc.).
- Attention-Based Encoding: A shared Multilayer Perceptron (MLP) encodes each joint's data. A softmax attention mechanism with a learnable temperature then aggregates these variable-sized joint inputs into a single, fixed-size latent vector. This allows the network to dynamically route relevant joint information regardless of how many joints the robot has.
- Universal Morphology Decoding: To output actions, a universal decoder pairs the core network's action latent vector back with each individual joint's description vector to sample joint-specific motor commands.
The entire architecture is trained end-to-end using Proximal Policy Optimization (PPO) combined with a time-dependent penalty curriculum to smoothly shape the robots' gaits over time.
Datasets
The policy was trained entirely in the CPU-based MuJoCo physics simulator using a diverse suite of 16 simulated robots across 48 parallel environments:
- 9 Quadrupeds: Including ANYmal B/C, Google Barkour v0/vB, MAB Silver Badger, Petoi Bittle, and Unitree A1/Go1/Go2.
- 5 Humanoids: PAL Robotics Talos, Robotis OP3, SoftBank Nao V5, Unitree G1, and Unitree H1.
- 1 Biped: Agility Robotics Cassie.
- 1 Hexapod: A custom 18-joint hexapod.
For real-world validation, the policy was deployed on physical hardware using the Unitree A1, MAB Honey Badger, and MAB Silver Badger.
Key Insights and Ideas
- Conditioning on Physics, Not Just State: By feeding the network explicit joint parameters (like friction, damping, and gear ratios), the policy doesn't just memorize a gait; it learns to reason about how a joint will react based on its physical constraints.
- Attention as a Dynamic Router: The attention mechanism acts as a universal translator. It allows the network to automatically adjust its internal representations whether it is balancing a top-heavy humanoid or coordinating a stable six-legged hexapod gait.
- The Multi-Task Advantage: Counterintuitively, training on 16 diverse robots simultaneously yields higher final performance and better sample efficiency than training individual, specialized policies for each robot from scratch. The shared representation space accelerates learning across different morphologies.
Key Results
- Superiority Over Baselines: URMA consistently outperformed standard padding and multi-head MTRL baselines in both training efficiency and final reward across the 16-robot fleet.
- Zero-Shot Transfer: When evaluated on a simulated Unitree A1 withheld from training, URMA achieved immediate, optimal locomotion performance, vastly outperforming single-robot policies trained for the same duration.
- Out-of-Distribution Generalization: URMA successfully controlled the MAB Silver Badger—a robot featuring an extra spine joint in its trunk and lacking foot sensors—outperforming baselines and maintaining a stable gait through zero-shot transfer and subsequent few-shot fine-tuning.
- Real-World Deployment: The exact same URMA policy trained in simulation was successfully deployed zero-shot onto three physical quadruped robots, enabling robust walking over pavement, grass, and turf. Crucially, this included the MAB Honey Badger, which was entirely absent from the simulation training set.
Limitations and Future Work
While URMA represents a major step toward a "foundation model" for locomotion, the authors note several limitations:
- Blind Locomotion: The current framework omits exteroceptive sensors (like cameras or LiDAR), meaning the robots cannot navigate complex obstacles or uneven terrain requiring vision.
- Zero-Shot Humanoid Transfer: Zero-shot transfer remains highly challenging for humanoids due to their inherent instability and the limited number of bipedal platforms in the training set. Expanding the diversity of the training fleet and incorporating advanced data augmentation are suggested avenues to bridge this gap.
Summary updated