Abstract
Deep Reinforcement Learning techniques are achieving state-of-the-art results in robust legged locomotion. While there exists a wide variety of legged platforms such as quadruped, humanoids, and hexapods, the field is still missing a single learning framework that can control all these different embodiments easily and effectively and possibly transfer, zero or few-shot, to unseen robot embodiments. We introduce URMA, the Unified Robot Morphology Architecture, to close this gap. Our framework brings the end-to-end Multi-Task Reinforcement Learning approach to the realm of legged robots, enabling the learned policy to control any type of robot morphology. The key idea of our method is to allow the network to learn an abstract locomotion controller that can be seamlessly shared between embodiments thanks to our morphology-agnostic encoders and decoders. This flexible architecture can be seen as a potential first step in building a foundation model for legged robot locomotion. Our experiments show that URMA can learn a locomotion policy on multiple embodiments that can be easily transferred to unseen robot platforms in simulation and the real world.
Introduction
In legged robotics, controllers are typically specialized "brittle" systems. A policy trained for a specific quadruped cannot be easily transferred to a humanoid or even a different quadruped with a different joint configuration. This challenge, known as the correspondence problem, arises because every robot possesses a unique number of joints (action space) and sensors (observation space). Traditional neural network architectures are often hard-coded for a specific morphology, and common workarounds like zero-padding or multi-head architectures scale poorly and fail to generalize.
Researchers have developed URMA (Unified Robot Morphology Architecture) to address this. URMA is a single reinforcement learning policy capable of controlling 16 diverse robot types simultaneously—ranging from bipeds and quadrupeds to hexapods. This approach aims to create a "foundation model" for locomotion that understands the underlying physics of movement regardless of the specific robot body.
Methods: The URMA Architecture
The core innovation of URMA lies in its use of morphology-agnostic encoders and decoders. Instead of raw joint indices, the network processes Joint Descriptions—fixed-size vectors containing physical metadata such as rotation axes, relative positions, torque limits, and gear ratios.
The architecture consists of three main components:
- The Encoder: Utilizes an attention mechanism to aggregate information from an arbitrary number of joints and feet into a single, fixed-size latent vector. This allows the network to route relevant information dynamically based on the robot's complexity.
- The Core: A standard Multi-Layer Perceptron (MLP) that processes the abstract latent representation to determine high-level locomotion strategies.
- The Decoder: A universal morphology decoder that maps the core's output back to specific motor commands for each joint, guided by the joint descriptions.
The system is trained using Proximal Policy Optimization (PPO) with extensive Domain Randomization (DR) to ensure robustness against real-world variations in friction, mass, and motor strength.
Experimental Results
The policy was trained in the MuJoCo physics simulator on a suite of 16 distinct robots, including various quadrupeds (ANYmal, Unitree), humanoids (Talos, G1, H1), a biped (Cassie), and a hexapod.
Key Findings
- Knowledge Transfer: Training on multiple morphologies simultaneously accelerated learning. URMA reached higher performance levels faster than policies trained on single robots, suggesting the discovery of a "shared language" of locomotion.
- Zero-Shot Generalization: URMA demonstrated impressive zero-shot transfer to the Unitree A1 (withheld from training), performing as well as specialized policies trained for 1.6 billion steps.
- Robustness: The attention-based encoder makes the policy naturally robust to sensor failures. In tests where foot observations were dropped, URMA maintained stability while traditional baselines failed.
- Real-World Deployment: The same single policy file successfully controlled three different physical robots (Unitree A1, MAB Honey Badger, and MAB Silver Badger) across various terrains like grass and pavement.
Discussion and Future Work
While URMA represents a significant step toward a foundation model for locomotion, several limitations remain. The current model is "blind," relying solely on proprioception; integrating exteroceptive sensors like LiDAR or cameras is essential for obstacle avoidance. Furthermore, while the policy controls humanoids, zero-shot transfer to new humanoid shapes is more challenging due to their inherent instability. Future work will focus on expanding the training set to hundreds of morphologies to further enhance generalization.
Summary updated