CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation

Choo Sin Wai, Dake Zhong, Jingye Zhang, Zhifang Liu, Siyao Xiao, Zirong Ding, Yuhong Zhang, Kangjie Chen, Dongbin Zhang, Guorui Song, Guocun Wang, Peng Jiao, Haoqian WangarXiv 2026 · arXiv

TL;DR

CIALLO (Cross-embodiment Intent All In One Space) is a novel action codec that enables robots to share manipulation knowledge by mapping actions, visual changes, and language into a unified "intent space." By separating embodiment-specific control from high-level behavioral intent, CIALLO allows a new robot to be calibrated in just 30 minutes using only 10 seconds of data, significantly outperforming traditional cross-embodiment transfer methods.

What Problem Does This Paper Address?

In robotics, "knowledge" is often trapped within a specific body. A grasp demonstrated by a seven-jointed arm at 30Hz looks fundamentally different in data than the same grasp performed by a six-jointed arm at 5Hz. Current cross-embodiment learning attempts to pool data from different robots, but they often struggle because recorded actions are "entangled" with the specific coordinates and control conventions of the hardware.

The authors identify a critical gap: action reconstruction alone doesn't guarantee that similar behaviors across different robots share a common representation. To truly transfer skills, we need a way to represent what a robot intends to do (e.g., "pick up the bottle") independently of how its specific motors must turn to achieve it.

Methods

The core of the paper is the CIALLO Action Codec. It functions as a bridge between a robot's physical signals and a shared, abstract intent space.

  1. Shared Intent Space: CIALLO uses three separate encoders to map different modalities into the same bottleneck representation ($z$):
  2. Action Encoder ($Enc_A$): Processes action chunks and proprioception.
  3. Visual Encoder ($Enc_V$): Looks at the visual change between current and future frames.
  4. Text Codec ($Enc_T$): Processes natural language instructions.
  5. Alignment via Contrastive Learning: The model is trained using InfoNCE losses to ensure that an action, its visual effect, and its linguistic description all land in the same spot in the intent space.
  6. Embodiment Conditioning: To translate intent back into robot-specific commands, the shared decoder ($Dec_A$) is conditioned on an Embodiment Essential Vector ($E$) and arm/axis embeddings. This vector captures the unique "signature" of a robot's control interface.
  7. TransConv Head: Instead of simple regression, CIALLO uses a structured decoding head that predicts actions as a softmax expectation over 256 bins. This provides a more robust and efficient decoding basis than standard flow-matching or diffusion heads.
Figure 1. Overview of CIALLO. (A) During pretraining, three encoders map an action chunk with proprioception, a pair of current and future frames, and an instruction grounded in the current frame into a shared intent space. We align their outputs with contrastive losses and train a shared decoder to reconstruct the demonstrated actions from each intent, conditioned on the robot’s embodiment parameters. (B) To learn a policy, Qwen-CIALLO predicts intent from observations and instructions, using the frozen action encoder as a teacher and the demonstrated actions to supervise the decoder output. CS denotes cosine similarity loss. (C) To transfer the codec to a new robot, we first calibrate its essential vector and arm and axis embeddings while keeping the shared weights fixed, then train the policy with the calibrated codec frozen.

Datasets

The authors pretrained CIALLO on a massive mixture of 125,434 episodes across eight different embodiments, including:

  • Real-world data: Bridge (WidowX) and Fractal (Google Robot).
  • Simulation data: LIBERO (Panda) and five different robots from the RoboTwin 2.0 benchmark (Aloha AgileX, Franka Dual, ARX X5, Piper, and UR5).

For evaluation, they tested on the standard LIBERO suites and the challenging RoboTwin 2.0 leaderboard, as well as real-world deployments on SO-101 and Piper robots.

Key Insights and Ideas

  • Separation of Calibration and Task Learning: CIALLO treats "learning the body" and "learning the task" as two distinct phases. Once the codec is pretrained, adapting to a new robot only requires fitting the small essential vector $E$, leaving the complex task-prediction policy frozen.
  • Visual Cues as a Universal Anchor: The authors found that visual intent is much less sensitive to embodiment mismatches than action intent. This suggests that the visual effects of an action serve as a stable "ground truth" that different robots can agree on.
  • The "More is Better" Effect: Interestingly, performance on the original robots (like the Panda) actually improved as more diverse embodiments were added to the pretraining set. Diversity in hardware helps the model learn a more generalized and robust intent space.
  • Efficiency Matters: By using a single-pass expectation readout, the CIALLO decoder is significantly faster than the iterative flow-matching heads used in models like $\pi_0$.
Figure 2. The action codec. The action encoder (middle) uses the essential vector E to map control signals into shared intent, incorporating context from the proprioceptive encoder (top). The decoder (bottom) combines this intent with the target action schema and current proprioception, remodulates the result for the target embodiment, and uses the TransConv Head to predict each control scalar on a shared basis of 256 ordered bins. The decoded action is the softmax expectation over the bin centers.

Key Results

  • Rapid Adaptation: CIALLO can adapt to a completely new robot body using a recording of at most 10 seconds, with calibration taking only ~30 minutes on a single GPU.
  • State-of-the-Art Performance:
    • On LIBERO, Qwen-CIALLO achieved a 97.3% average success rate, surpassing major baselines like OpenVLA (76.5%) and StarVLA (96.6%).
  • On RoboTwin 2.0, it reached 52.7% success, outperforming the previous best co-trained model (StarVLA at 46.5%).
  • Real-World Success: In real-robot tests, CIALLO achieved 76.7% success across SO-101 and Piper robots, notably outperforming a fully finetuned $\pi_0$ (70.0%) despite using a frozen codec.
  • Inference Speed: The CIALLO decoder (JAX version) is 13.6x faster than the flow-matching head used in $\pi_{0.5}$, making it highly suitable for real-time control.

Limitations and Future Work

While CIALLO excels at cross-robot transfer, the authors note that the current policy still requires a small number of task-specific demonstrations (around 20) after calibration. They aim to move toward more "zero-shot" task transfer in the future.

Another promising direction is human-to-robot transfer. The authors plan to align human egocentric videos (showing how humans manipulate objects) with robot actions in the shared intent space, potentially allowing robots to learn directly from massive libraries of human video data.

Summary updated

Abstract

Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces. Action reconstruction alone does not ensure that similar behaviors across robots share a representation. To address this gap, we introduce CIALLO, an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of short-horizon behavioral intent. An embodiment-conditioned decoder maps this intent back to robot-specific actions, allowing a shared representation to support different control interfaces. After pretraining on eight embodiments, CIALLO adapts to a new robot from one recording of at most 10 seconds by updating only its embodiment embeddings. The calibrated codec then remains frozen while a policy learns to predict intent from observations and instructions. We instantiate this policy with Qwen3.5-4B to form Qwen-CIALLO, whose backbone requires no large-scale robot-action pretraining before policy finetuning. With 20 demonstrations per task, Qwen-CIALLO achieves 76.7\% average success across real SO-101 and single-arm Piper robots, compared with 70.0\% for fully finetuned $π_0$. In simulation, we additionally finetune the action decoder, achieving 97.3\% success on LIBERO and 52.7\% on RoboTwin 2.0. The LIBERO policy also achieves 80.8\% on LIBERO-Plus without further finetuning. Together, these results support a shared action codec as a transferable control interface that separates embodiment calibration from task learning.
BibTeX
@misc{wai2026ciallo,
  author = {Choo Sin Wai and Dake Zhong and Jingye Zhang and Zhifang Liu and Siyao Xiao and Zirong Ding and Yuhong Zhang and Kangjie Chen and Dongbin Zhang and Guorui Song and Guocun Wang and Peng Jiao and Haoqian Wang},
  title = {{CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation}},
  year = {2026},
  eprint = {2611.13732},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13732},
}
Figure