VSpark: Speculative Decoding for Autoregressive Vision-Language-Action Models

Suho Yoo, Youngjoon Jang, Hyebin Cho, Taeyeop Lee, Jaewook Jeong, Joon Son ChungarXiv 2026 · arXiv

TL;DR

VSpark is a new speculative decoding framework that speeds up autoregressive Vision-Language-Action (VLA) models by up to 2.10× without sacrificing any accuracy. By cleverly reusing rejected predictions and aligning the "thoughts" of a small draft model with a large target model, VSpark overcomes the unique challenges of robotics—where similar physical motions can be represented by wildly different discrete tokens.

What Problem Does This Paper Address?

Autoregressive VLAs are powerful because they treat robot control as a "next-token prediction" task, inheriting the reasoning capabilities of large language models. However, this comes at a cost: speed. Generating actions one token at a time is slow, often too slow for real-time robot control.

Standard speculative decoding tries to fix this by using a tiny "draft" model to guess several tokens ahead, which a large "target" model then verifies in a single parallel pass. In text generation, this works great. In robotics, it hits a wall. Robot actions are often discretized into tokens. Because many different token sequences can represent nearly identical physical motions, a draft model might propose a sequence that is physically "correct" but token-wise "wrong." Standard systems would reject this draft entirely, wasting the computation.

Furthermore, modern VLAs use "compressed tokenizers" where one token might represent a chunk of time or multiple joints. This makes it nearly impossible to use "relaxed" verification (accepting "close enough" tokens) without losing the mathematical guarantee that the output matches the original large model.

Methods

VSpark introduces three core design choices to make speculative decoding viable for lossless robot control:

  1. Rollouts with Calibrated Action Noise: Draft models are usually trained on static expert data. VSpark instead trains the draft model on "noisy" rollouts. It injects Gaussian noise—calibrated to match the draft model's own prediction errors—into the target model's actions during simulation. This exposes the draft model to the diverse, slightly-off-track states it will actually encounter during real-world inference.
  2. Feature Alignment: Instead of just training the draft model to predict the next discrete token, VSpark forces it to regress the target model's internal hidden features. This provides a continuous, high-dimensional signal that helps the draft model learn the underlying motion the target model is "thinking" about, rather than just memorizing labels.
  3. Continuation Reuse: This is the "aha!" moment of the paper. When the target model rejects a drafted token, it doesn't just say "no"; it also computes what should have come next. VSpark realizes that if the rejected draft was physically similar to the correct action, the target's predictions following that rejection are likely still useful. It recycles these predictions into the next drafting cycle instead of throwing them away.
Overview of VSpark's training and inference pipeline, highlighting noise injection, feature alignment, and continuation reuse.

Datasets

The authors evaluated VSpark across a wide range of benchmarks and VLA architectures:

  • LIBERO: A standard suite of 40 manipulation tasks (Goal, Spatial, Object, Long).
  • LIBERO-Plus & LIBERO-Pro: Stress-tests involving distribution shifts in lighting, viewpoints, and object layouts.
  • SimplerEnv (WidowX): A real-world-to-sim benchmark using a different robot embodiment and data from BridgeData V2.
  • Target Models: They tested on four distinct VLAs ($\pi_0$-FAST, StarVLA-FAST, ActionCodec-AR, and G0.5) covering both frequency-based and learned vector-quantized tokenizers.

Key Insights and Ideas

  • Token Distance $\neq$ Action Distance: The paper demonstrates that two tokens being "close" in embedding space is a poor proxy for the robot's physical actions being similar. This justifies why VSpark sticks to exact verification rather than relaxing the rules.
  • The "Similarity" Benefit: In robotics, a rejected branch in a search tree is often "nearly right." By reusing continuations, VSpark finds that 45% to 55% of predictions made along rejected paths are actually correct once the prefix is fixed.
  • Efficiency over Depth: Interestingly, the authors found that adding more layers to the draft model (making it "smarter") actually decreased total speedup because the overhead of running a larger draft model outweighed the gains in token acceptance. A single-layer draft model with tree verification was the "sweet spot."
Illustration of continuation reuse: predictions along rejected branches are recycled for the next cycle.

Key Results

VSpark consistently outperformed five state-of-the-art speculative decoding baselines (like EAGLE-3 and DSpark) across all tested models and environments:

  • Speedup: Achieved 1.28× to 2.10× faster inference compared to standard autoregressive decoding.
  • Acceptance Length: Increased the average number of tokens accepted per cycle (e.g., from 3.16 to 3.94 on $\pi_0$-FAST).
  • Lossless Performance: Because verification remains strict, the success rate of the robots remained identical to the original, unaccelerated models (within numerical precision limits).

Limitations and Future Work

  • Recurrent Architectures: Models with complex recurrent states (like G0.5) are harder to accelerate because parallel tree verification becomes computationally expensive.
  • Real-World Training: While VSpark works on real-robot data, collecting the "noisy rollouts" required for training currently relies on having a simulator. Developing ways to collect this data safely on physical hardware is a key next step.
  • Language Model Limits: The authors noted that "continuation reuse" provides less benefit for pure language tasks (like GSM8K) compared to robotics, suggesting this technique is specifically tuned to the continuous nature of physical motion.

Summary updated

Abstract

Our aim is to accelerate autoregressive vision-language-action models (VLAs), which generate robot actions one token at a time. Speculative decoding makes this possible: a lightweight draft model proposes several tokens at once, and the VLA verifies them in a single pass, accepting only those that match its own predictions, so decoding remains lossless. The speedup, however, is limited in robot control, where many different tokens encode nearly the same motion and a draft close to the correct action is still rejected. To address this, our approach, VSpark, is built upon three design choices: (i) rollouts collected with calibrated action noise, which expose the draft to a wider range of robot states; (ii) feature alignment, which trains the draft to match the VLA's hidden features rather than its output tokens alone; and (iii) continuation reuse, which keeps the predictions the VLA has already computed along rejected drafts as candidates for the next verification step, instead of discarding them. On four VLAs with different action tokenizers and four manipulation benchmarks, VSpark achieves 1.28x to 2.10x speedup while keeping the VLA's decoding lossless.
BibTeX
@misc{yoo2026vspark,
  author = {Suho Yoo and Youngjoon Jang and Hyebin Cho and Taeyeop Lee and Jaewook Jeong and Joon Son Chung},
  title = {{VSpark: Speculative Decoding for Autoregressive Vision-Language-Action Models}},
  year = {2026},
  eprint = {2611.22666},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22666},
}
Figure