Large multimodal models often fail at visual math and diagram reasoning in a specific way. They describe the figure correctly, then partway through a long chain of reasoning they state a "fact" about the image that the image does not support. Each step reads as confident and fluent, and the model never looks back. We propose a test-time check that asks, at every reasoning step, a simple counterfactual question: would the model have written this step if the image were blurred beyond recognition? If a step introduces new visual content, such as a number, an object, or a spatial relation, but its probability barely changes when the image is destroyed, then the step came from the language model's priors and not from the picture. We flag that step, roll it back, and regenerate it while favoring candidates that the image actually supports.
This matters because the usual uncertainty signal, the entropy of the model's next-token distribution (how flat or peaked its predictions are), is blind to exactly this failure. Prior-driven hallucinations are typically low entropy. The model is sure there is a sink in the kitchen, or that the chart title gives the answer. A per-step measure of how much the image changes the model's predictions can separate "confident because it saw it" from "confident because it guessed." Our central claim is that this measure localizes reasoning-induced hallucinations better than plain entropy or attention-based signals. We further claim that intervening only at the flagged steps improves accuracy on visual reasoning benchmarks at lower cost than applying anti-hallucination decoding to every token.
Core Claim and Mechanism
Let $I$ be the image, $\tilde I$ a heavily blurred copy, and $x_{<t}$ the question plus the reasoning generated so far. For each generated token $y_t$ we compute the pointwise visual gain:
$$g_t = \log p(y_t \mid I, x_{<t}) - \log p(y_t \mid \tilde I, x_{<t}).$$
For a reasoning step $s$, let $F_s$ be its set of fresh content tokens: numerals, nouns, and spatial or relational words that appear neither in the question nor in any earlier step. These tokens must come either from the image or from the model's priors, because they cannot be copied from text already in context. The step score is the average gain over these tokens:
$$G_s = \frac{1}{|F_s|}\sum_{t \in F_s} g_t.$$
Steps with no fresh content tokens are pure deduction, such as arithmetic on numbers already stated. They are never gated, which avoids the obvious false positive of penalizing legitimate symbolic steps that should not depend on the image. The gate fires when $G_s$ falls below $\tau$. We set $\tau$ as a quantile of the running distribution of $G_s$ on the unlabeled test stream, not as a fixed constant. This responds directly to the known brittleness of fixed entropy thresholds in test-time adaptation. When the gate fires, we discard the step, sample $k$ alternative continuations, and keep the one with the highest $G_s$ among those whose plain entropy is not abnormally high. Both scores come from the same two forward passes, so selection adds no further passes beyond sampling the $k$ candidates.
The method borrows its contrast from Selective Training for Large Vision Language Models via Visual Information Gain. That paper measures the loss drop between clear and blurred images and uses it to prune text-dominant tokens during offline supervised training of LLaVA-1.5 on descriptive tasks. We move the same quantity to inference, apply it to unlabeled multi-step reasoning chains, and use it as a decision rule instead of a loss mask. These are different uses of the signal: VIG never decides anything during generation.
Why This Gap Is Real
Several lines of evidence motivate the proposal. MATH-Vision attributes 42.2% of GPT-4V errors to reasoning rather than recognition. MIRAGE builds a benchmark from cases where the description is correct but the answer is wrong. It also shows that correcting only the reasoning chain recovers the right answer about 70% of the time, so the errors live in identifiable intermediate steps. More Thinking, Less Seeing? shows that attention drifts from visual tokens toward earlier reasoning tokens as chains lengthen. Hallucination therefore develops mid-chain rather than at the start. Why LVLMs Are More Prone to Hallucinations in Longer Responses finds the same drift in long-form description.
Other work constrains the design. POPE, HallusionBench, and Plot Twist together show sharp predictions driven by co-occurrence statistics, "Yes"-bias, and chart titles. This is why any entropy-only gate, however finely applied, will pass confidently wrong steps. The test-time adaptation literature makes the same point from the other side. EATA filters on low entropy and works well on corrupted ImageNet, where confident usually means correct. That assumption is exactly what fails under strong language priors.
The closest existing method is SDR-MCoT. It checks the entropy of the first token to decide whether to reason at all, then reshapes attention maps to emphasize grounded regions. It makes no decision after the chain starts, and its signal is plain entropy. Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning tracks entropy throughout the chain using a running change-point detector (CUSUM, a cumulative-sum test for a shift in a sequence's mean), but only on text reasoning, where there is no image to ignore. Functional Attention Control and SHARP intervene inside the network with fixed gains or steering vectors applied uniformly. They provide intervention machinery, but no rule for when to intervene. Our contribution is that rule: a per-step, training-free, image-counterfactual trigger. We show its value with a deliberately simple intervention, resampling, so the trigger can be evaluated on its own.
Research Program
The work has three parts. The first is a measurement study. We take reasoning chains from Qwen2.5-VL-7B (the base model in MIRAGE) and Ocean-R1-7B (the RL-trained reasoning model that scored best on RH-Bench). We annotate a subset of MIRAGE and MathVista chains at the step level, roughly 300 chains with two annotators, marking whether each step's visual claims are supported by the image. On these labels we compare how well each candidate signal ranks hallucinated steps above faithful ones, reported as area under the ROCcurve. We also report the subset of hallucinated steps whose plain entropy is in the bottom half, the "confidently wrong" steps, since this is where we predict the visual-gain score will separate from entropy most clearly.
The second part tests the gate-and-resample procedure on full benchmarks at matched compute. The third part runs ablations. These cover the choice of degraded image (heavy blur, mean-color image, or no image), the fresh-content-token restriction versus averaging over all tokens, and a quantile threshold versus a fixed threshold.
Cost needs attention. The blurred-image pass roughly doubles forward compute during generation. Both conditions share the text prefix and can run as a batch of two, so latency grows less than twofold. All comparisons are therefore reported against generated tokens and total forward FLOPs, not against raw accuracy alone.
Baselines
The naive reference point is the same model decoding greedily with a standard "think step by step" prompt. This tells us whether any intervention helps. It is not a strawman: MATH-Vision reports that chain-of-thought can lower accuracy for some multimodal models, so we also report direct answering.
The strongest training-free competitor on the gating question is SDR-MCoT (Tu et al.). It reports a 63.69% average across eight benchmarks on Qwen2-VL-7B, against 57.34% for standard chain-of-thought. We run its first-token gate and attention enhancement on Qwen2-VL-7B and Qwen2.5-VL-7B over our benchmark subset, using the same prompts and the same threshold-tuning budget as our method: a single held-out calibration split of 200 items per benchmark. Beating it means higher accuracy on MIRAGE and HallusionBench at no more forward FLOPs. We also report the stack of SDR-MCoT followed by our step gate, since the two are compatible.
The strongest per-step entropy competitor is CUSUM change-point gating from Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning, ported to multimodal chains and paired with the same resample-on-flag intervention. That paper reports 63.06% average accuracy with 11.1% fewer tokens on text benchmarks. No reported result was found on Lacuna for CUSUM on any multimodal benchmark. This baseline is the key ablation, because it holds the intervention fixed and swaps only the trigger. If plain-entropy CUSUM matches visual gain on step-detection AUROC and end accuracy, our central claim is false.
For the strongest overall reference on MIRAGE we cite Logos (MIRAGE). It improves Qwen2.5-VL-7B from 28.8% to 37.1% accuracy and LHS from 0.5996 to 0.6568, using curriculum RL training plus an external hint LLM. It is not a matched comparison, because it trains the model and adds a second model at inference. We report it as an upper reference and test whether our training-free gate closes a meaningful share of that 8.3-point gap. For MathVista, Socratic-MCTS (Socratic-MCTS) reports 0.740 to 0.782 via tree search. We run a FLOP-matched version of it where feasible. Uniform anti-hallucination decoding is represented by RITUAL (RITUAL), which reports 88.87% versus 84.13% on POPE for LLaVA-1.5. We apply it to every token in the chain at equal compute.
Why not just…?
Why not just use plain per-step entropy, which needs no second forward pass? This is the most serious threat, and the measurement study is built to settle it. The evidence against it is that prior-driven errors are confident by construction. POPE reports near-99% "Yes" rates, and Plot Twist finds accuracy near 79% when the title contains the answer, falling to the 30% range when it does not. Both describe low-entropy behavior. The decisive number is detection AUROC on the confidently-wrong subset. If plain entropy matches visual gain there, we drop the second pass and report the cheaper method.
Why not just gate on the fraction of attention going to image tokens? More Thinking, Less Seeing? identifies attention drift as the mechanism, and attention is free to read. But attention measures where the model looks, not whether what it sees changes what it says. SDR-MCoT needs morphological erosion just to suppress attention sinks before its maps are usable. We include the visual-attention ratio, with and without sink suppression, as a gate in both the AUROC study and the end-task runs. If it performs as well, the honest conclusion is that the cheaper signal suffices.
Why not just apply visual contrast to every token, as RITUAL or contrastive decoding do? Uniform intervention pays the extra cost on every step, including pure deduction, where it can hurt. RITUAL notes that some transformations degrade certain domains. SHARP and FAC report contrastive-decoding latency of 13.97s versus 7.04s and slowdowns of up to 6.6×. The FLOP-matched comparison shows directly whether selective intervention wins. If uniform contrast is equally accurate at equal cost, the gate adds nothing.
Why not just sample many chains and take a majority vote? Voting assumes errors are independent. Prior-driven hallucinations are systematic, the "consistent garbage" that EMPO filters against. Socratic-MCTS reports beating 32-sample self-consistency. We run self-consistency with as many samples as our method's total FLOPs allow, and also as a reranker using mean $G_s$. If voting matches us on HallusionBench, where systematic bias dominates, the prediction fails.
Proposed Experiments and Decisive Checks
We evaluate on MIRAGE, using accuracy, the logical-hallucination rate, and LHS. We also use MathVista and MATH-Vision (accuracy, plus the Transformation Geometry subset, where GPT-4o falls to about 18% and spatial reasoning dominates). HallusionBench is scored by question-pair accuracy, where GPT-4V reaches 31.42%. RH-Bench is scored by RH-AUC, where Ocean-R1-7B reports 0.63 as the best result, to check that improving reasoning does not cost perception.
The first decisive check is localization. On the annotated MIRAGE steps, visual gain must beat plain entropy, CUSUM, and the attention ratio in AUROC on the confidently-wrong subset. If it does not, the mechanism is wrong regardless of downstream accuracy.
The second check is intervention value at matched FLOPs. Gate-and-resample must improve MIRAGE accuracy over SDR-MCoT and over CUSUM with the same resampling, and must not lower RH-AUC.
The third check is the fresh-token ablation. Averaging gain over all tokens should flag many pure-arithmetic steps and hurt MATH-Vision accuracy. That result would confirm that separating "claims about the image" from "deduction" is the ingredient doing the work.
References
- Selective Training for Large Vision Language Models via Visual Information Gain
- Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM
- More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
- Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context
- Evaluating Object Hallucination in Large Vision-Language Models
- Hallusionbench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
- Plot Twist: Multimodal Models Don't Comprehend Simple Chart Details
- Efficient Test-Time Model Adaptation without Forgetting
- Mitigating Low-Quality Reasoning in MLLMs: Self-Driven Refined Multimodal CoT with Selective Thinking and Step-wise Visual Enhancement
- Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning
- Mitigating Hallucination in Multimodal Reasoning via Functional Attention Control
- SHARP: Steering Hallucination in LVLMs via Representation Engineering
- Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions
- RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs
- Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization