Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse

Jiafeng Liang, Xianhao Ou, Tao Ren, Rongchuan Mu, Runxuan Liu, Dandan Tu, Ming Liu, See-Kiong Ng, Bing QinarXiv 2026 · arXiv

TL;DR

Reinforcement Learning with Verifiable Rewards (RLVR) can inadvertently train video models to "cheat" by relying on textual shortcuts rather than visual evidence. This paper introduces Bifocal Self-distillation Policy Optimization (BSPO), a framework that uses self-generated reasoning principles and visual perturbations to ensure models remain grounded in the video while improving their reasoning capabilities.

What Problem Does This Paper Address?

Large Multimodal Models (LMMs) are increasingly trained using RLVR—a paradigm where models are rewarded simply for getting the right answer (outcome-level optimization). While effective for text, the authors discover a critical failure mode in video reasoning: Shortcut Amplification.

Because many video questions can be guessed using common sense or linguistic patterns (e.g., "If a team scores, the stadium probably cheers"), RLVR often encourages the model to ignore the video and double down on these "textual priors." The authors characterize this as Perceptual Uncertainty Collapse: as training progresses, the model becomes more confident, but that confidence is increasingly decoupled from the actual visual evidence.

Figure 1: Training dynamics observations of video RLVR. (a) The illustration of the two evaluation sets. (b) Accuracy of selecting the ground-truth (GT) answer on the faithful set and the shortcut answer on the counterfactual set.

Methods

To combat this, the authors propose BSPO, which moves beyond simple "right/wrong" rewards by reweighting the learning process based on two new signals:

  1. Principle-guided Credit Assignment (PCA): Instead of treating every word in a correct answer as equally important, the model is asked to generate "reasoning principles" by comparing its own successful and failed attempts. It then gives more "credit" to tokens that align with these logical principles.
  2. Shortcut-aware Contrastive Perturbation (SCP): This is the "grounding" check. The system creates a "counterfactual" version of the video (e.g., by shuffling frames or cropping). If the model’s prediction remains the same even when the visual evidence is destroyed, it’s a sign of shortcut reliance. BSPO penalizes this invariance, forcing the model to be sensitive to the actual video content.

These two signals are combined into a "bifocal" objective that adaptively modulates the RL updates, ensuring they are both logically sound and visually grounded.

Datasets

The models were trained on 20,000 QA pairs sampled from the Video-R1-260k dataset. Evaluation was conducted across six diverse benchmarks to ensure a comprehensive look at video reasoning:

  • Long Video: Video-MME and LVBench.
  • General Video: TempCompass and MVBench.
  • Complex Reasoning: Video-MMMU and MMVU.

The authors also utilized a custom Counterfactual Set, where key video segments were replaced with visually irrelevant but semantically related clips to test for shortcut reliance.

Key Insights and Ideas

  • The Entropy Gap: The authors use "visual-conditioning entropy gap" ($\bar{G}$) to measure how much the video actually helps the model. In standard RL, this gap shrinks over time, meaning the video becomes less relevant to the model's decision-making.
  • Gradient Misalignment: Through a clever "mechanistic analysis," they show that RL updates often align with directions that reduce uncertainty based on text, while the alignment with visual-based uncertainty reduction actually declines.
  • Self-Correction via Principles: The model can act as its own teacher. By summarizing why it got an answer right or wrong, it creates a feedback loop that refines its own internal credit assignment.
Figure 2: Perceptual uncertainty analysis of video RLVR. (a) RL training reduces the video-conditioned entropy ¯Hv, while the visual-conditioning entropy gap ¯G also decreases, indicating that increased confidence is not accompanied by stronger visual evidence utilization. (b) The decline of visual-specific compression ¯K and geometric misalignment, suggests that such sharpening is progressively decoupled from visual entropy-reducing directions and increasingly dominated by text-prior compression.

Key Results

BSPO consistently outperformed standard RLVR baselines (like GRPO and DAPO) and other self-distillation methods across all benchmarks.

  • Performance Gains: On the Qwen3-VL-4B backbone, BSPO achieved a +6.2% improvement on MMVU and +5.7% on TempCompass compared to the base model.
  • Visual Faithfulness: Unlike standard RL, which saw a narrowing gap between faithful and counterfactual tests, BSPO actually enlarged this gap. This proves the model became more robust to shortcuts and more dependent on actual visual cues.
BSPO reverses the shortcut amplification trend, widening the gap between faithful and counterfactual performance as training progresses.

Limitations and Future Work

The authors acknowledge that BSPO relies on the model's inherent ability to generate meaningful reasoning principles. For very small or weak models, these self-generated "principles" might be low-quality, potentially limiting the effectiveness of the PCA component.

Future work could explore more sophisticated video perturbation techniques beyond shuffling and cropping, or investigate how to scale this "bifocal" approach to even larger multimodal models where the risk of complex, high-level shortcuts is even greater.

Summary updated

Abstract

Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective post-training paradigm for enhancing the video reasoning capability of Large Multimodal Models (LMMs). Despite its empirical effectiveness, we show that outcome-level optimization can introduce an undesirable shortcut amplification effect, whereby models increasingly favor text-prior answers even when they are not supported by the visual evidence. To characterize this behavior, we investigate the training dynamics of video RLVR through the lens of perceptual uncertainty, analyzing how predictive uncertainty evolves under video-conditioned and text-only contexts. Our analysis identifies a perceptual uncertainty collapse phenomenon, in which RL training continuously sharpens the model's answer distribution while the uncertainty reduction attributable to visual evidence progressively diminishes. A further gradient-direction analysis reveals that such confidence enhancement becomes increasingly less aligned with visually grounded uncertainty reduction and is instead dominated by text-prior compression. Motivated by these findings, we propose Bifocal Self-distillation Policy Optimization (BSPO), a self-distillation framework that regulates policy updates with two complementary distributional signals. Specifically, BSPO derives empirical reasoning principles from intra-group correct and incorrect rollouts to identify reasoning-sensitive tokens, and introduces shortcut-aware visual perturbation to penalize predictions that remain invariant when task-relevant visual evidence is weakened. By adaptively reweighting the original RL advantage, BSPO preserves the task-solving benefits of RLVR while promoting updates that are both reasoning-sensitive and visually grounded. Extensive experiments validate the effectiveness of BSPO in improving video reasoning performance and strengthening visual faithfulness.
BibTeX
@misc{liang2026uncovering,
  author = {Jiafeng Liang and Xianhao Ou and Tao Ren and Rongchuan Mu and Runxuan Liu and Dandan Tu and Ming Liu and See-Kiong Ng and Bing Qin},
  title = {{Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse}},
  year = {2026},
  eprint = {2610.37211},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37211},
}
Figure