TL;DR
CIALLO (Cross-embodiment Intent All In One Space) is a novel action codec that enables robots to share manipulation knowledge by mapping actions, visual changes, and language into a unified "intent space." By separating embodiment-specific control from high-level behavioral intent, CIALLO allows a new robot to be calibrated in just 30 minutes using only 10 seconds of data, significantly outperforming traditional cross-embodiment transfer methods.
What Problem Does This Paper Address?
In robotics, "knowledge" is often trapped within a specific body. A grasp demonstrated by a seven-jointed arm at 30Hz looks fundamentally different in data than the same grasp performed by a six-jointed arm at 5Hz. Current cross-embodiment learning attempts to pool data from different robots, but they often struggle because recorded actions are "entangled" with the specific coordinates and control conventions of the hardware.
The authors identify a critical gap: action reconstruction alone doesn't guarantee that similar behaviors across different robots share a common representation. To truly transfer skills, we need a way to represent what a robot intends to do (e.g., "pick up the bottle") independently of how its specific motors must turn to achieve it.
Methods
The core of the paper is the CIALLO Action Codec. It functions as a bridge between a robot's physical signals and a shared, abstract intent space.
- Shared Intent Space: CIALLO uses three separate encoders to map different modalities into the same bottleneck representation ($z$):
- Action Encoder ($Enc_A$): Processes action chunks and proprioception.
- Visual Encoder ($Enc_V$): Looks at the visual change between current and future frames.
- Text Codec ($Enc_T$): Processes natural language instructions.
- Alignment via Contrastive Learning: The model is trained using InfoNCE losses to ensure that an action, its visual effect, and its linguistic description all land in the same spot in the intent space.
- Embodiment Conditioning: To translate intent back into robot-specific commands, the shared decoder ($Dec_A$) is conditioned on an Embodiment Essential Vector ($E$) and arm/axis embeddings. This vector captures the unique "signature" of a robot's control interface.
- TransConv Head: Instead of simple regression, CIALLO uses a structured decoding head that predicts actions as a softmax expectation over 256 bins. This provides a more robust and efficient decoding basis than standard flow-matching or diffusion heads.
Datasets
The authors pretrained CIALLO on a massive mixture of 125,434 episodes across eight different embodiments, including:
- Real-world data: Bridge (WidowX) and Fractal (Google Robot).
- Simulation data: LIBERO (Panda) and five different robots from the RoboTwin 2.0 benchmark (Aloha AgileX, Franka Dual, ARX X5, Piper, and UR5).
For evaluation, they tested on the standard LIBERO suites and the challenging RoboTwin 2.0 leaderboard, as well as real-world deployments on SO-101 and Piper robots.
Key Insights and Ideas
- Separation of Calibration and Task Learning: CIALLO treats "learning the body" and "learning the task" as two distinct phases. Once the codec is pretrained, adapting to a new robot only requires fitting the small essential vector $E$, leaving the complex task-prediction policy frozen.
- Visual Cues as a Universal Anchor: The authors found that visual intent is much less sensitive to embodiment mismatches than action intent. This suggests that the visual effects of an action serve as a stable "ground truth" that different robots can agree on.
- The "More is Better" Effect: Interestingly, performance on the original robots (like the Panda) actually improved as more diverse embodiments were added to the pretraining set. Diversity in hardware helps the model learn a more generalized and robust intent space.
- Efficiency Matters: By using a single-pass expectation readout, the CIALLO decoder is significantly faster than the iterative flow-matching heads used in models like $\pi_0$.
Key Results
- Rapid Adaptation: CIALLO can adapt to a completely new robot body using a recording of at most 10 seconds, with calibration taking only ~30 minutes on a single GPU.
- State-of-the-Art Performance:
- On LIBERO, Qwen-CIALLO achieved a 97.3% average success rate, surpassing major baselines like OpenVLA (76.5%) and StarVLA (96.6%).
- On RoboTwin 2.0, it reached 52.7% success, outperforming the previous best co-trained model (StarVLA at 46.5%).
- Real-World Success: In real-robot tests, CIALLO achieved 76.7% success across SO-101 and Piper robots, notably outperforming a fully finetuned $\pi_0$ (70.0%) despite using a frozen codec.
- Inference Speed: The CIALLO decoder (JAX version) is 13.6x faster than the flow-matching head used in $\pi_{0.5}$, making it highly suitable for real-time control.
Limitations and Future Work
While CIALLO excels at cross-robot transfer, the authors note that the current policy still requires a small number of task-specific demonstrations (around 20) after calibration. They aim to move toward more "zero-shot" task transfer in the future.
Another promising direction is human-to-robot transfer. The authors plan to align human egocentric videos (showing how humans manipulate objects) with robot actions in the shared intent space, potentially allowing robots to learn directly from massive libraries of human video data.
Summary updated