VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding

Yu-Cheng Chou, Shihao Wang, Jin Huang, Chi-Pin Huang, Chiao-An Yang, Xianzhe Fan, Jing Wang, Zhiqi Li, Yuanguo Kuang, Ruijie Zheng, Johan Bjorck, Sifei Liu, Hang Su, Abhishek Badki, Lei Zhang, Alan Yuille, Linxi "Jim'' Fan, Jan Kautz, Zhiding YuarXiv 2026 · arXiv

Physical AI agents, like autonomous cars and humanoid robots, need to do more than just talk; they need to plan their movements in space. While Vision-Language Models (VLMs) have become the "brains" of these systems, they suffer from a significant bottleneck: they are incredibly slow at spatial reasoning.

Standard VLMs generate spatial plans—like a robot arm's trajectory or a bounding box—one coordinate token at a time. For a complex 3D path, this can mean hundreds of sequential forward passes, leading to high latency that makes real-time reaction impossible.

VFlash, a new framework from NVIDIA and academic partners, changes the game by treating spatial plans as structured "primitives" rather than just strings of text.

TL;DR

VFlash accelerates visual planning in VLMs by decoding entire spatial primitives (like trajectories or bounding boxes) in a single parallel pass. By prioritizing structural validity over exact token-level agreement with a target model, it achieves up to a 25x speedup and improved spatial accuracy for robotics and autonomous driving tasks.

What Problem Does This Paper Address?

The core issue is the autoregressive (AR) bottleneck. When a VLM predicts a trajectory, it might output something like <point_1_x> <point_1_y> .... Each token requires a full pass through the massive model. As we demand higher resolution or longer-horizon plans, the time it takes to "think" grows linearly.

Existing solutions like speculative decoding try to guess multiple tokens at once, but they usually fail if the guess doesn't match the target model's output exactly. In spatial tasks, a coordinate that is off by a single millimeter is a "wrong token" to a standard decoder, even if it’s a perfectly valid and accurate plan for a robot.

Methods

VFlash introduces Flash Primitive Decoding (FPD). The system consists of a large, frozen "target" VLM and a lightweight, shallow "drafter" model.

  1. Tool Calling & Schema: The process starts with a tool call that defines the structure of the output (e.g., "I need a 10-point 3D trajectory"). This fixes the role of every upcoming token slot before decoding even begins.
  2. Parallel Drafting: Instead of going token-by-token, the lightweight drafter predicts the entire primitive (all coordinates and delimiters) in one single forward pass.
  3. Relaxed Acceptance: This is the "secret sauce." Unlike standard speculative decoding, VFlash does not require the draft to match the target VLM's tokens exactly. As long as the draft is "structurally valid" (e.g., coordinates are within bounds, delimiters are in the right place), it is accepted in full.
  4. Verification for Text: For non-spatial output (like sub-task descriptions), VFlash falls back to standard verified speculative decoding to ensure linguistic quality.
The VFlash design uses tool calls to trigger parallel decoding of structured visual primitives.

Datasets

The authors evaluated VFlash across a massive range of tasks:

  • 2D Trajectories: ShareRobot-Bench and VABench-V.
  • 3D Robot Trajectories: InternA1, AgiBot, and MolmoBot.
  • Bounding Boxes: COCO and Dense200.
  • Dense 3D Motion: PointMotionBench (including HOT3D, WorldTrack, and DAVIS).
  • Downstream Applications: MolmoSpaces (manipulation), LIBERO (robotics benchmarks), and autonomous driving suites (NAVSIM, Waymo).

Key Insights and Ideas

  • Spatial Accuracy $\neq$ Token Agreement: The authors discovered that a lightweight drafter trained on annotated spatial data can actually be more accurate than the large AR target model. Forcing the drafter to match the AR model's exact (and sometimes noisy) token sequence actually hurts performance.
  • System-1 vs. System-2 Thinking: VFlash treats spatial primitives as "System-1" tasks—fast, intuitive, and parallel. It reserves the "System-2" (slow, deliberative) AR process for complex language reasoning.
  • Boundary Alignment: By ensuring the drafter predicts a complete geometric object (like a whole box) rather than a fixed window of tokens, the model avoids the "splitting" issues that plague other parallel decoders.

Key Results

VFlash delivers massive improvements in both speed and success rates:

  • Latency Reduction: On edge hardware like the NVIDIA Jetson Thor, VFlash reduces decoding latency by approximately 80% for manipulation tasks.
  • Massive Speedups: For dense 3D point motion forecasting, VFlash is roughly 25x faster than autoregressive decoding.
  • Improved Success Rates: In the MolmoSpaces manipulation suite, a world action model conditioned on VFlash plans achieved a 70.2% average success rate, compared to 62.6% for the baseline.
  • Better Accuracy: Across 3D trajectory tasks, VFlash consistently reduced prediction errors (ADE/FDE) compared to the AR target.
Latency comparison across different hardware platforms shows VFlash significantly outperforming AR and DFlash baselines.
Success rates on the LIBERO benchmark family show that VFlash (green) consistently outperforms the baseline and versions without visual primitives.

Limitations and Future Work

While VFlash is a major step forward, the authors note a few areas for growth:

  • Target Pass Requirement: Even though the draft is accepted without token matching, the target VLM still needs one pass to update its internal cache and provide context for the next part of the response.
  • Single Primitive Focus: The current work focuses on drafting one primitive at a time. Future iterations could explore generating multiple complementary primitives (e.g., multiple object trajectories) simultaneously.
  • Data Dependency: The drafter's performance is heavily dependent on the quality of the annotated spatial data used for training.

VFlash proves that for Physical AI, we don't need to wait for models to "talk" through their plans. By embracing the inherent structure of the physical world, we can make robots that think—and act—much faster.

Summary updated

Abstract

Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.
BibTeX
@misc{chou2026vflash,
  author = {Yu-Cheng Chou and Shihao Wang and Jin Huang and Chi-Pin Huang and Chiao-An Yang and Xianzhe Fan and Jing Wang and Zhiqi Li and Yuanguo Kuang and Ruijie Zheng and Johan Bjorck and Sifei Liu and Hang Su and Abhishek Badki and Lei Zhang and Alan Yuille and Linxi "Jim'' Fan and Jan Kautz and Zhiding Yu},
  title = {{VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding}},
  year = {2026},
  eprint = {2611.08449},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08449},
}
Figure