Fewer Tokens, More Answers: Verifiable-Reward Tuning of an Ornith-Style VLM for Multi-View Driving QA

T N Rickesh, Pranav Vinodh, Sowmya Kamath S

TL;DR

Reasoning-heavy Vision-Language Models (VLMs) often fail in real-time applications because they "overthink," hitting token limits before delivering an answer. This paper demonstrates that tuning a model with verifiable rewards (GRPO) acts as a powerful test-time compute compression operator, reducing average response length by over 50% while maintaining accuracy by resolving the "answer-collection bottleneck."

What Problem Does This Paper Address?

In autonomous driving, decisions must be made in milliseconds. While modern VLMs can perform complex reasoning (Chain-of-Thought), they often pay a "deliberation tax": they generate verbose, step-by-step narrations of camera frames that consume the entire generation budget.

If a model has a 256-token limit and spends all of them describing the scene without ever outputting the final multiple-choice answer, it’s an operational failure. The authors investigate whether post-training can force models to be more concise—delivering the same reasoning quality but in fewer tokens—to fit within the strict latency and compute budgets of edge deployment.

Methods

The researchers focused on Ornith, a reasoning-enabled VLM derivative. They applied Group Relative Policy Optimization (GRPO), a reinforcement learning technique, using a verifiable reward function.

Unlike subjective rewards, their function was strictly binary and rule-based:

  1. Correctness: Did it get the right answer? (+2.0)
  2. Format: Is it in the requested JSON format? (±0.5)
  3. Extractability: Can a simple parser find the answer? (±0.25)

They compared the base model against the GRPO-tuned adapter across different token caps (256 vs. 512 tokens) and tested a "recovery" protocol where the model was prompted with "Therefore, the answer is" if it failed to provide one initially.

Datasets

The study utilizes WaymoQA, a multi-view visual question-answering dataset derived from real-world driving sequences. The evaluation was performed on:

  • A 400-question "screen": Used to compare learning rates and initial gains.
  • A 2,511-question development set: Used for broader validation of the "concise delivery" hypothesis.

Key Insights and Ideas

  • The Answer-Collection Bottleneck: Most "failures" at low token budgets aren't due to poor vision or logic; they are due to token exhaustion. The base model was often on the right track but literally ran out of space to speak.
  • Post-Training as Compression: The most striking finding is that GRPO didn't necessarily make the model "smarter" at spatial reasoning. Instead, it taught the model to compress its verbalization.
  • The Crossover Effect: When the researchers gave the base model a larger budget (512 tokens) or a nudge to finish its thought, the performance gap between the base and the tuned model nearly vanished. This proves the tuning's primary value is efficiency, not raw reasoning capacity.
  • Latent vs. Explicit Reasoning: The authors argue that for robotics, we should move toward models that deliberate in "latent space" (hidden representations) rather than emitting verbose text chains that clog up the inference pipeline.
This figure shows how GRPO shifts the model from "unextractable" (token-exhausted) states to "strict, correct" answers.

Key Results

  • Accuracy Boost at Low Budgets: On a 256-token cap, GRPO increased accuracy from 55.50% to 63.75% (+8.25 percentage points).
  • Massive Token Reduction: On the full development set, the tuned pipeline reduced the mean generated tokens from 108.6 to 47.1—a 56.6% reduction in emitted length.
  • Vanishing Gains: When the budget was expanded to 512 tokens and "recovery" was enabled, the accuracy gain dropped to 0.00 pp on the screen set. This confirms that the base model already had the "knowledge," but the tuned model had the "concision."
The left panel shows how the accuracy gain disappears as the budget increases, while the right panel highlights the significant reduction in mean generated tokens.

Limitations and Future Work

The authors note several limitations:

  • Selection Bias: The most impressive gains were reported on the same "screen" used to select the best-performing adapter.
  • Single Model Focus: The results are specific to the Ornith checkpoint and may not automatically transfer to other architectures like Bonsai without further testing.
  • Lack of Latency Data: While token counts are a good proxy, the paper does not provide end-to-end wall-clock latency or FLOP measurements.

Future work should focus on "fingerprinting" these reasoning derivatives to see if they benefit from similar reward structures and exploring whether these models actually consume all eight camera views provided in the prompt.

Summary updated

Abstract

Reasoning-enabled model derivatives must turn intermediate deliberation into a usable answer within a deployment budget. We study that transition in an Ornith checkpoint adapted to multi-camera driving question answering with verifiable rewards. Paired archived outputs show that, on a selected 400-question development screen with a 256-token cap, GRPO increases answer extraction from 78.25% to 90.25% and accuracy from 55.50% to 63.75%. Of 39 wrong-to-correct transitions, 33 begin with an unextractable base response. The base's 87 unextractable responses all reach the token cap. Checkpoint-verified reconstruction on those same questions gives a much smaller first-pass accuracy gain at 512 tokens: +0.75 percentage points, versus +8.25 at 256. A later answer-cued continuation is analyzed separately; it helps the base more and narrows the adapter difference. On the full 2,511-question development set, this recovery-enabled pipeline reduces mean generated tokens from 108.6 to 47.1, while the paired accuracy interval includes zero gain. We frame these findings as adaptation of answer delivery under finite budgets, with explicit selection and provenance limits. Ornith's self-scaffolding lineage and newer reasoning-enabled derivatives such as Bonsai motivate the problem, but our measured results concern Ornith alone and do not establish a shared architectural mechanism or cross-model transfer.
BibTeX
@misc{rickesh2026fewer,
  author = {T N Rickesh and Pranav Vinodh and Sowmya Kamath S},
  title = {{Fewer Tokens, More Answers: Verifiable-Reward Tuning of an Ornith-Style VLM for Multi-View Driving QA}},
  year = {2026},
  eprint = {2611.00833},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00833},
}
Figure