Watching Itself Watch: Self-Auditing Visual Reliance for Video Reasoning

Rohan Dugad, Mi Luo, Kristen GraumanarXiv 2026 · arXiv

TL;DR

VIOS (Visually-Aware On-Policy Self-Distillation) is a post-training framework that enables video-reasoning models to "audit" their own visual reliance, identifying exactly which tokens in a reasoning chain are under-utilizing visual evidence. By focusing learning on these "visually decisive" moments, it achieves state-of-the-art performance with a staggering 60x reduction in compute and 111x fewer generated tokens compared to traditional reinforcement learning methods.

What Problem Does This Paper Address?

Modern Multimodal Large Language Models (MLLMs) are increasingly capable of "reasoning" through videos, but they often fall into a trap: they rely on textual shortcuts and linguistic patterns rather than actually looking at the video. For example, if asked "In what order did the person pack the bag?", a model might guess "laptop last" simply because that's a common pattern in its training text, even if the video shows something different.

Existing post-training methods like Group Relative Policy Optimization (GRPO) try to fix this by rewarding correct answers. However, these methods are "sparse"—they provide a single reward for a long chain of thought—and "expensive," requiring many model rollouts and often a larger "judge" model (like GPT-4) to grade the results. The authors argue that video reasoning hinges on just a few "visually decisive" tokens, and current methods fail to identify or specifically train those critical moments.

Methods

The authors introduce VIOS, which allows a model to "watch itself watch." The core idea is Self-Auditing: a model can determine how much it relies on a video by comparing its prediction for the next word with the video versus without it. If the prediction changes significantly, the model is relying on vision; if not, it's relying on text.

Figure 1: Overview of VIOS. A single on-policy rollout provides per-token, vision-aware supervision through three JSD comparisons between the student pS and EMA teacher pT : distillation toward the teacher, contrastive separation from the student’s no-vision counterpart, and a reliance gap gn that emphasizes tokens where the teacher uses vision and the student does not.

Key components include:

  • EMA Teacher: A co-evolving version of the model (updated via Exponential Moving Average) that acts as a teacher. Crucially, the teacher is "answer-privileged"—it sees the ground-truth answer, allowing it to provide a high-quality target for the student to match.
  • Contrastive Objective: A loss term that actively pushes the student model to make its predictions different from what they would be if the video were missing, forcing it to utilize visual evidence.
  • Reliance-Gap Gate: This is the "secret sauce." It calculates the difference between the teacher's visual reliance and the student's. If the teacher is using vision for a specific token but the student isn't, the gate "opens wide," concentrating the training signal on that specific, under-utilized moment.
  • On-Policy Efficiency: Unlike GRPO, which requires 8+ rollouts per prompt, VIOS works with a single rollout, drastically cutting costs.

Datasets

The researchers trained VIOS using the Video-R1-260k corpus. They evaluated the system across 12 diverse benchmarks to ensure broad capability:

  • General: MVBench, Video-MME, MINERVA.
  • Knowledge-Intensive: MMVU, Video-MMMU, VideoMathQA, SciVideoBench.
  • Temporal Reasoning: TempCompass, TVBench.
  • Long-Video/Spatial: LongVideoBench, MLVU, VSI-Bench.

They also tested on static image benchmarks (ChartQA, BLINK, CV-Bench, MMMU) to demonstrate that the "visual reliance" principle generalizes beyond video.

Key Insights and Ideas

  • The Model is its Own Best Auditor: You don't need a massive external model to tell a student it's ignoring the video. The internal divergence between "vision-on" and "vision-off" states provides a dense, per-token supervision signal that is much more informative than a simple "correct/incorrect" at the end of a sentence.
  • Dense vs. Sparse Supervision: By targeting specific tokens (like the name of an object or a specific action), VIOS fixes reasoning errors where they happen, rather than hoping the model figures it out through trial and error.
  • Co-evolution over Stagnation: Using an EMA teacher that improves alongside the student prevents the "rationalization ceiling" that occurs when distilling from a fixed, mediocre base model.

Key Results

VIOS consistently outperformed base models and contemporary systems like Video-R1 and VideoChat-R1 across almost all benchmarks.

  • Performance Gains: It achieved average gains of 3.6 to 4.6 points across three different model backbones (Qwen2.5-VL and InternVL2.5).
  • Extreme Efficiency: Compared to Video-R1 (a leading GRPO-based system), VIOS required:
  • 9x fewer training steps.
  • 60x fewer total FLOPs (compute).
  • 111x fewer generated tokens during training.
Comparison of training costs between VIOS and GRPO-based methods.

Qualitatively, the "Reliance-Gap Gate" successfully highlights tokens where the model previously failed to ground its reasoning in the video, such as identifying specific objects being packed or detecting whether an object actually fell off a table.

Limitations and Future Work

  • Initialization Dependency: VIOS requires a base model that can already generate a coherent "chain of thought" (usually achieved via initial Supervised Fine-Tuning). It cannot yet elicit reasoning from a raw, "base" model.
  • Answer-Key Requirement: The current setup requires ground-truth answers for training, which limits its use for purely open-ended, unlabelled video data.
  • Final Token Fragility: The authors noted a rare failure mode where the "answer token" at the very end of a chain can be overly sensitive to the contrastive loss, occasionally flipping a correct reasoning chain into a wrong final answer.
  • Future Directions: The authors suggest applying this "self-auditing" principle to other modalities like audio or depth sensors, and exploring ways to relax the requirement for ground-truth answers.

Summary updated

Abstract

Video reasoning hinges on a small number of visually decisive tokens embedded in otherwise largely language-driven chains of thought. Yet prevailing post-training methods such as GRPO overlook this structure, assigning coarse trajectory-level rewards while requiring multiple rollouts and often costly external judges. We introduce Visually-Aware On-Policy Self-Distillation (VIOS), a post-training framework that identifies where a model under-utilizes visual evidence and concentrates learning at those positions. Our key insight is that a model can audit its own visual reliance: the divergence between its next-token distributions with and without the video reveals how strongly each prediction depends on the visual input. Measuring this reliance for both a student and an answer-privileged teacher exposes a per-token reliance gap, identifying positions where the teacher exploits visual evidence that the student overlooks. VIOS translates this signal into dense, targeted supervision through three complementary components: an exponential moving average (EMA) teacher that co-evolves with the student, a contrastive objective that encourages predictions to depend on the video, and a reliance-gap gate that focuses learning on visually under-utilized tokens. Crucially, VIOS requires only a single on-policy rollout from a single model, without external rewards, judges, or larger teachers. Across 12 benchmarks spanning general, temporal, spatial, knowledge-intensive, and long-video understanding, VIOS consistently improves three model backbones and outperforms contemporary video-reasoning systems, while training in over 9x fewer steps, 60x fewer FLOPs, and 111x fewer generated tokens. Project page: https://vision.cs.utexas.edu/projects/VIOS/
BibTeX
@misc{dugad2026watching,
  author = {Rohan Dugad and Mi Luo and Kristen Grauman},
  title = {{Watching Itself Watch: Self-Auditing Visual Reliance for Video Reasoning}},
  year = {2026},
  eprint = {2611.11949},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11949},
}
Figure