TL;DR
Current video AI models excel at multiple-choice questions about unusual or counterfactual events, but they often "cheat" by exploiting text clues in the question options. This paper introduces FADE, a two-stage training framework that forces models to actively discover and explain visual anomalies on their own as textual hints are progressively stripped away.
What Problem Does This Paper Address?
Counterfactual video understanding tests whether an AI truly grasps physical laws and common sense—for instance, recognizing the absurdity of a rocket suddenly surfacing from the water instead of a jet ski.
However, the authors point out a major flaw in how we currently evaluate these models: almost all benchmarks rely on Multiple-Choice Questions (MCQs). These questions and candidate options act as "textual anchors" that inadvertently leak the target event. Instead of autonomously discovering an anomaly, the model simply has to perform text-guided verification.
When you remove these textual options—moving from an MCQ to an open-ended question or a simple prompt to "describe the video"—even state-of-the-art models like GPT-5.6 experience massive performance drops. They fail because they don't actually know where or what the anomaly is without a text hint to guide their attention.
Methods
To bridge this gap, the authors present FADE (Fading Textual Anchors for Counterfactual Discovery and Explanation), an evidence-first, two-stage training paradigm:
- Stage I: Evidence-Internalized Supervised Fine-Tuning (SFT): This stage forces the model to ground its internal representations in actual visual evidence. By using a lightweight projector and a dual-branch re-grounding mechanism, the model learns to map its hidden response states directly to the specific video frames where the physical anomaly occurs. This ensures the model is actually "looking" at the right evidence.
- Stage II: Fading-Anchor Reinforcement Learning (RL): Once the model is visually grounded, it undergoes a "scaffold-and-consolidate" curriculum. The model is fed the same video across three progressively harder formats: MCQ $\rightarrow$ Open-ended Question Answering (OQA) $\rightarrow$ Captioning. Using Group Reward-Decoupled Normalization Policy Optimization (GDPO), the model is rewarded for maintaining correct answers even as the text anchors completely fade away.
Additionally, the authors designed an automated evaluation pipeline that reformulates existing MCQ datasets into aligned OQA and captioning tasks, removing the need for expensive manual data curation.
Datasets
The framework was trained and evaluated on two primary datasets:
- DualityVidQA: The training split (containing over 104,000 factual and counterfactual video samples) was used for SFT and RL training. The test split was used to evaluate performance across all three text-fading levels.
- IPV-Bench: A benchmark consisting exclusively of counterfactual videos, used to test the model's generalization capabilities under reduced textual guidance.
Key Insights and Ideas
- The MCQ Illusion: High multiple-choice accuracy in video models is often an illusion sustained by text-based hints rather than true visual understanding.
- Task-Agnostic Grounding: By forcing internal hidden states to align with visual anomaly timestamps during SFT, the model develops an intrinsic ability to spot weird events regardless of how it is prompted.
- Curriculum Fading: Rather than training on unconstrained tasks from scratch, progressively "fading" the text anchors allows the model to transfer its text-guided reasoning capabilities into fully autonomous discovery.
Key Results
FADE (built on top of the open-source Qwen3-VL-8B baseline) demonstrates remarkable robustness against the removal of textual hints, outperforming proprietary models like GPT-5.6 on open-ended tasks.
- High Performance Retention: On the DualityVidQA-test set, FADE retains 90.4% of its MCQ performance when transitioned to Open QA, and 67.4% when transitioned to pure captioning. For comparison, GPT-5.6 only retains 48.1% and 30.7% respectively.
- Absolute Scores: On IPV-Bench captioning (where no question is provided), FADE achieves an accuracy score of 60.2%, vastly outperforming GPT-5.6's 25.5% and the baseline Qwen3-VL-8B's 16.1%.
Limitations and Future Work
While FADE marks a significant step toward unconstrained video understanding, it has a few limitations:
- Dependence on Temporal Annotations: Stage I (SFT) requires precise timestamp annotations of when the counterfactual event occurs in the training data.
- LLM-as-a-Judge Evaluation: Evaluating open-ended answers and captions requires a frozen semantic LLM verifier, which can introduce minor evaluation biases.
- Future Directions: The authors suggest extending this fading-anchor strategy to other multimodal domains where models suffer from text-bias, such as complex visual document reasoning or long-form video summarization.
Summary updated