IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation

Shijie Lian, Bin Yu, Xiaopeng Lin, Zhaolong Shen, Laurence Tianruo Yang, Yurun Jin, Haishan Liu, Changti Wu, Hang Yuan, Cong Huang, Kai ChenarXiv 2026 · DOI

TL;DR

This paper introduces IntentVLA, a novel framework for robot manipulation that addresses the challenge of "short-horizon intent ambiguity." By conditioning action generation on recent visual history, IntentVLA significantly improves the consistency and stability of robot actions in scenarios where current observations alone are insufficient to determine the correct next step.

What Problem Does This Paper Address?

Modern vision-language-action (VLA) models are powerful, but they often struggle with a subtle but critical problem: short-horizon intent ambiguity under partial observability. Imagine a robot performing a complex task. While the overall goal might be clear, at any given moment, the robot might be in a state where the current visual input and instruction could lead to several different, plausible next actions. This is especially true when the robot's actions are chunked (broken down into discrete steps).

Existing VLA models, which typically condition on the current frame and instruction, can get "stuck" or "confused" in these ambiguous situations. They might pick one plausible intent, execute a chunk, and then, upon replanning, switch to a different plausible intent based on the same visual cues, leading to jerky, inconsistent, or even failed executions. Human demonstrators, on the other hand, maintain a consistent short-horizon intent within an episode, even if they could have chosen differently. The problem isn't multimodality itself, but the uncommitted multimodality under aliased observations.

Methods

IntentVLA tackles this by introducing a history-conditioned imitation learning framework. The core idea is to leverage recent visual history to infer the robot's current short-horizon intent.

Here's how it works:

  1. History Encoding: A frozen VGGT-based encoder processes a window of recent visual observations. This encoder extracts "history evidence tokens" and a compact "summary token" that captures geometric information and inter-frame relations.
  2. Intent Information Fusion: This history evidence is then fused with the current visual-language context (from a Qwen3-VL backbone) using gated cross-attention. This fusion process enriches the current observation with information about the episode's recent commitment.
  3. Intent-based Action Generation: The fused context, along with the compact history-evidence summary token, forms a "short-horizon intent representation." This representation is then used to condition a standard Diffusion Transformer (DiT)-based flow-matching action head, which predicts the robot's next action chunk.

Crucially, IntentVLA does not explicitly infer a discrete intent label. Instead, the history-conditioned representation implicitly guides the action generation towards a consistent short-horizon intent.

Datasets

  • AliasBench: A newly introduced benchmark built on top of RoboTwin2. It comprises 12 manipulation tasks specifically designed to isolate short-horizon observation aliasing. AliasBench includes matched simulation training data and evaluation environments, focusing on scenarios where current frames are visually ambiguous but recent history provides disambiguation.
  • SimplerEnv: Used for evaluating the generalizability of IntentVLA on four WidowX manipulation tasks.
  • LIBERO: A benchmark for evaluating policies across multiple suites, with a focus on longer, multi-stage manipulation routines.
  • RoboCasa-GR1: A benchmark with 24 diverse manipulation tasks involving articulated objects and varied geometries.
  • Real-world snack-cleanup task: A dual-arm Franka Research 3 setup used for real-world validation.

Key Insights and Ideas

  • History is Key for Consistency: The most significant insight is that simply conditioning on the current frame is insufficient for robust, consistent robot behavior in ambiguous situations. Recent visual history provides the crucial context for maintaining a stable short-horizon intent.
  • AliasBench as a Diagnostic Tool: The creation of AliasBench is a valuable contribution. It moves beyond simply measuring task success to specifically evaluate a policy's ability to maintain consistent decisions when faced with visually similar but intent-distinct states.
  • Implicit Intent is Sufficient: The paper demonstrates that explicitly inferring and supervising discrete "intent labels" is not necessary. A history-conditioned representation can implicitly guide the policy towards consistent intent, simplifying the learning process.
  • History Fusion Benefits Geometric Reasoning: The results suggest that the VGGT history encoder is particularly beneficial for tasks requiring fine-grained geometric understanding and alignment, where subtle changes in viewpoint or object pose over time are critical.

Key Results

  • AliasBench Performance: IntentVLA significantly outperforms strong VLA baselines on AliasBench, achieving a 45.8% average success rate, a 36.8% improvement over the Qwen3-VL-GR00T baseline. Gains are largest on crossing-path and back-and-forth tasks.
  • Standard Benchmark Performance: IntentVLA also shows strong performance on SimplerEnv (72.9% average success, outperforming previous best by 4.7 points), LIBERO-Long (97.4% success), and RoboCasa-GR1 (57.0% average success).
  • Inter-Chunk Consistency: IntentVLA reduces inter-chunk inconsistency (measured by ICC-L2) by 17.6% on average in ambiguous windows, indicating more stable and aligned action sequences. This improvement is consistent across all ambiguity families.
  • Real-World Success: In a real-world snack-cleanup task, IntentVLA improved the expected number of placed snacks per rollout from 1.44 to 1.86 and was the only model to collect all four snacks within the budget.

Limitations and Future Work

  • Short-Horizon Focus: IntentVLA's strength lies in recovering short-horizon intent from recent visual history. It is not designed for general episodic memory or long-horizon memory, limiting its application scope to locally aliased manipulation. Tasks requiring remembering sparse events or recovering from large closed-loop deviations may require different approaches.
  • No Explicit Long-Horizon Planner: While IntentVLA improves consistency in longer routines (e.g., LIBERO-Long), it does not introduce an explicit long-horizon planner.
  • Future Work: The authors suggest that longer-term memory or planning modules would be needed for tasks requiring memory of sparse events or recovery from significant deviations. The benchmark and code are released to encourage further research in this area.

Summary updated

Abstract

Robot imitation data are often multimodal: similar visual-language observations may be followed by different action chunks because human demonstrators act with different short-horizon intents, task phases, or recent context. Existing frame-conditioned VLA policies infer each chunk from the current observation and instruction alone, so under partial observability they may resample different intents across adjacent replanning steps, leading to inter-chunk conflict and unstable execution. We introduce IntentVLA, a history-conditioned VLA framework that encodes recent visual observations into a compact short-horizon intent representation and uses it to condition chunk generation. We further introduce AliasBench, a 12-task ambiguity-aware benchmark on RoboTwin2 with matched training data and evaluation environments that isolate short-horizon observation aliasing. Across AliasBench, SimplerEnv, LIBERO, and RoboCasa, IntentVLA improves rollout stability and outperforms strong VLA baselines
BibTeX
@misc{lian2026intentvla,
  author = {Shijie Lian and Bin Yu and Xiaopeng Lin and Zhaolong Shen and Laurence Tianruo Yang and Yurun Jin and Haishan Liu and Changti Wu and Hang Yuan and Cong Huang and Kai Chen},
  title = {{IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation}},
  year = {2026},
  eprint = {2605.14712},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.14712},
}
Figure