TL;DR
OneTwoVLA unifies high-level reasoning and low-level robot control into a single vision-language-action model. By adaptively switching between reasoning and acting modes, it eliminates the latency and coordination issues of traditional dual-system approaches while excelling at long-horizon planning and error recovery.
What Problem Does This Paper Address?
In robotic manipulation, combining high-level cognitive reasoning (planning, error detection) with low-level physical execution (motor control) is a major challenge. Inspired by Kahneman’s behavioral economics framework, recent approaches often deploy a "dual-system" architecture: a slow, internet-pretrained Vision-Language Model (VLM) acts as System Two (high-level reasoning), passing instructions down to a fast Vision-Language-Action (VLA) model acting as System One (low-level execution).
However, this explicit separation introduces two critical flaws:
- Lack of Mutual Awareness: System Two might plan an action that System One is physically incapable of executing.
- High Latency: VLMs are slow. In dynamic real-world environments, a delayed response from System Two can result in outdated or irrelevant guidance, causing execution failures.
OneTwoVLA solves this by unifying both systems into a single model that dynamically decides when to think and when to act.
Methods
OneTwoVLA instantiates a single, unified network (built on top of the open-source $\pi_0$ base model) capable of autoregressively generating both textual reasoning and continuous action chunks.
The core innovation is Adaptive Inference, governed by two special decision tokens:
[BOR](Beginning of Reasoning): Triggers the model to enter reasoning mode and output natural language text (scene descriptions, high-level plans, historical summaries, and next-step instructions).[BOA](Beginning of Action): Triggers the model to enter acting mode and directly output continuous action values via an action expert architecture.
Instead of wasting computational resources reasoning at every single timestep, OneTwoVLA operates primarily in acting mode. It adaptively invokes reasoning only at critical junctures—such as completing a subtask, detecting an execution error, or receiving human intervention.
Datasets
To train this unified architecture, the authors curated two primary data sources:
- Reasoning-Enriched Robot Data: Human expert demonstrations segmented into reasoning intervals (annotated with scene descriptions, plans, and next steps via Gemini 2.5 Pro) and acting intervals. This includes 200 demonstrations for a "Tomato-Egg" cooking task, 600 for a "Hotpot" task, and 300 for a "Cocktail" mixing task.
- Synthetic Vision-Language Data Pipeline: To prevent the model from overfitting and improve generalization, the authors built a scalable pipeline. They used Gemini 2.5 Pro to generate tabletop descriptions and instructions, FLUX.1-dev to render high-quality images, and applied visual augmentations (like fisheye distortion and gripper overlays) to match real robot camera feeds. This produced 16,000 synthetic samples covering visual grounding and long-horizon tasks.
Key Insights and Ideas
- Cognitive Synergy Improves Motor Control: Explicitly training the model to reason doesn't just help with high-level planning; it actually improves low-level action accuracy. The authors found that action Mean Squared Error (MSE) on the validation set was 62% higher for a standard VLA ($\pi_0$) compared to OneTwoVLA.
- Probabilistic Reasoning Enables Multi-Modal Actions: When given an ambiguous command like "Grasp cube" in a scene with two identical cubes, standard VLAs often default to a single side due to regression averaging. Because OneTwoVLA reasons textually first ("I choose the left one to grasp"), its probabilistic language generation naturally allows it to sample and execute diverse, multi-modal actions cleanly.
- Synthetic Co-Training Unlocks Open-World Visual Grounding: Co-training on action-free, synthetic vision-language data allows the model to transfer abstract commonsense knowledge (e.g., understanding spatial relations or object attributes) directly into physical robot policies.
Key Results
The model was validated through extensive real-world robot experiments across long-horizon planning, error recovery, and human-robot interaction.
- Task Success & Efficiency: OneTwoVLA achieved an average success rate of 87% across three complex long-horizon tasks, outperforming the flat VLA baseline by 30% and the dual-system baseline by 24%.
- Reduced Latency: On the Tomato-Egg task, OneTwoVLA completed the objective in 184 seconds (with only 16 seconds spent on reasoning), nearly matching the speed of a non-reasoning flat VLA (176s) and drastically beating the dual-system approach (260s).
- Error Recovery: When failures occurred (e.g., losing grip on an oil bottle), OneTwoVLA successfully recovered 80% of the time, compared to ~57% for the baselines.
- Open-World Visual Grounding: In unseen environments with novel objects, co-training with synthetic data (OneTwoVLA-VL) boosted the visual grounding success rate from 8% to 73%.
Limitations and Future Work
Despite its strong performance, the authors note several limitations:
- Heuristic Triggers: The choice of when to switch to reasoning mode currently relies on hand-selected heuristics during training. Future work will explore Reinforcement Learning (RL) to let the model optimize its own reasoning schedule.
- Execution Pauses: The robot still pauses for 2–3 seconds when generating lengthy text. Developing asynchronous architectures that reason and act simultaneously is a promising next step.
- Inference Bottlenecks: As unified models scale up in parameter size, the computational cost of action inference could become a bottleneck, requiring advanced LLM acceleration techniques.
Summary updated