Tracing Where Visual Evidence Leaves a Multimodal Model's Reasoning

Vision-language models can describe a chart or a geometry diagram fluently and still get simple deductions about it wrong. The literature cannot yet say where inside the model the image stops mattering. This note proposes a direct measurement. Run the model twice on the same question and the same generated reasoning text, once with the real image and once with a minimally edited image whose correct answer differs (one bar taller, one angle wider, title unchanged). Then record, step by step and layer by layer, how far apart the two runs' internal states are. If the model actually uses what it sees, the two runs separate at the reasoning step where the edited value enters the argument and stay apart until the answer. If the model is answering from text cues, the two runs stay together. The result is a "visual dependence trace," and we plot it as paired trajectories in a shared low-dimensional view. Together these turn the vague label "perception-reasoning decoupling" into a measurable event with a location: the step and layer where image-driven differences in the model's internal state collapse.

The motivation is a disagreement in the evidence. MATH-Vision attributes 42.2% of GPT-4V's errors to reasoning and 31.9% to visual recognition, using manual error labels. Plot Twist shows chart accuracy reaching as high as 79% when the title contains the answer and falling to the 30% range when the title is removed. That suggests many apparently "correct reasoning" cases never consulted the image at all. Output-level analysis cannot separate three situations. In the first, the image was never encoded. In the second, it was encoded but dropped somewhere in the chain. In the third, it was carried all the way through but then reasoned about badly. The dependence trace separates them automatically and at scale.

Core Claim

Comparing hidden-state trajectories under answer-changing image edits localizes, per item, where visual evidence is lost along a multimodal reasoning chain. This localization does three things that single-trajectory geometry and output-level checks cannot do together. It sorts errors into "not seen / seen then dropped / used but misreasoned." It identifies correct answers that do not depend on the image. And, using a cheap masked-image variant, it flags likely errors on natural benchmarks.

Let $h_{t,l}$ be the hidden state at reasoning step $t$ and layer $l$ with the real image, and let $h'_{t,l}$ be the same state with the edited image, holding the generated text fixed (teacher forcing). We define the dependence

$$D(t,l) = \frac{\lVert h_{t,l} - h'_{t,l} \rVert}{\mathbb{E}\,\lVert h_{t,l} - \tilde h_{t,l} \rVert},$$

where $\tilde h_{t,l}$ is the state under an edit that leaves the answer unchanged, such as recoloring an irrelevant region. The ratio therefore measures sensitivity to answer-relevant evidence beyond generic sensitivity to any pixel change. We treat $D \approx 1$ as "no evidence carried" and look for the step at which $D$ falls to that level.

Relation to Prior Work

Two papers supply the trajectory machinery and motivate the move to multimodal inputs. Flowing Logics in Representation Space models chain-of-thought as a path through hidden-state space. It finds that the path's velocity and curvature, rather than its position, track the underlying logical operation, which it tests by shuffling step order. GeoR scores how stably representations settle across layers and uses that score to predict whether a reasoning model is correct; it includes Geometry3K as one evaluation set. Both analyze a single trajectory per input. A single trajectory can show that reasoning is unstable, but not whether it is grounded. A smooth, confident path driven by a chart title looks healthy to both methods. Our paired, counterfactual trajectory is the ingredient they lack. We reuse their velocity, curvature, and stability quantities as baselines rather than discarding them.

Two papers constrain the design. Hyperbolic Geometry of Reasoning shows that late layers of reasoning models compress. In that work a Euclidean probe's distortion reaches 0.562 against 0.090 for a hyperbolic probe, and plain Euclidean distances there can mislead. The normalization in $D$, which divides by a matched irrelevant-edit distance, is meant to cancel layer-specific scale changes such as the reported 41% norm drop. We also report a rank-based variant of $D$ in case normalization alone is insufficient. The same paper finds that sparse "thinking tokens" such as "Therefore" carry reasoning depth far better than average pooling (Spearman 0.871 vs 0.390). We therefore compute the trace at step-boundary tokens, not averaged over all tokens.

Two papers supply the intervention techniques and mark what our work does not do. VTI builds a visual steering direction from latent changes under patch masking and reduces object hallucination on POPE. That masking procedure is the source of our cheap test-time counterfactual. VTI, however, applies a fixed direction and evaluates existence hallucination, not multi-step reasoning. CraEG works in text-only math: it shows that probability concentrated on geometrically clustered tokens predicts failure, and it corrects sampling accordingly. It never looks at whether image evidence is present. The visualization literature, including the QA-LSTM interface and WildVis, renders static point clouds. Our display is a pair of trajectories with the dependence curve overlaid, and it is designed to show the moment where the two paths merge.

Research Program

The first component is a counterfactual stimulus set. Charts are rendered from data tables with a plotting library, following the elementary question types of Chart101 (axis extremes, point values, ranges). For each item we render a base chart and an answer-changing edit with all text fixed. We also render a title-conflict variant in which the title implies the original answer while the bars imply the new one; this reproduces Plot Twist's manipulation under our control. Geometry items are parametric diagrams with Geometry3K-style questions, where the edit changes one length or angle so that the answer changes. Each item also receives an irrelevant edit that serves as the normalizer in $D$.

The second component is the trace computation on open models: LLaVA-1.5, InstructBLIP, and Qwen-VL, the models VTI used, plus at least one stronger open reasoning model. For each item the model generates a chain under the real image. We then teacher-force that same chain under the edited image and under the irrelevant-edit image, and compute $D(t,l)$ at step-boundary tokens and at the visual tokens themselves.

The third component is the taxonomy. We train a linear probe on the visual-token states to read out the edited quantity. If the probe fails, the item is "not encoded." If the probe succeeds but $D$ collapses before the answer step, the item is "encoded then dropped." If $D$ stays high but the answer is wrong, the item is "used but misreasoned."

The fourth component is a test-time variant for natural benchmarks where controlled edits are impossible. Here the counterfactual is masking the image patches that the question refers to, or the whole image if the question gives no reference, following VTI's masking. This costs one extra teacher-forced pass and produces a "does this chain depend on the image?" score for error detection on MATH-Vision and GAOKAO-MM.

Baselines

The do-nothing reference is behavioral. We count an item as image-dependent if the final answer changes under the edit, and we predict correctness from model confidence alone. This tells us whether the answer depends on the image but not where the dependence goes. It also says nothing about wrong answers that flip for the wrong reason. Any claim of localization must add something beyond this flip rate.

The strongest existing method for error detection is GeoR. It is training-free, it adds negligible compute, and it already runs on a multimodal geometry set. The only number the file reports for it is 68.12% AUROC on GSM8K with R1-LLaMA-8B, which is text-only. No reported result was found on Lacuna for GeoR's AUROC on Geometry3K, MATH-Vision, or GAOKAO-MM. We will therefore rerun GeoR ourselves on the same models, items, and generated chains, using the same layers and the same held-out split for any calibration. Beating it means a higher AUROC on MATH-Vision and GAOKAO-MM for the masked-image dependence score, alone and in combination with GeoR, with a paired bootstrap showing the gain is not noise.

For the localization claim, the strongest competitor is single-trajectory geometry from Flowing Logics: curvature spikes or velocity drops along the real-image path. We will test whether these mark the step that our human annotators label as the point where visual information was dropped.

We add two further baselines that a reviewer would expect even though the file cites no paper for them. The first is the share of attention mass on image tokens at each step. The second is the linear readout probe applied at every step rather than only at the visual tokens.

The key ablation isolates the counterfactual ingredient. We replace the answer-changing edit with a random edit of equal pixel magnitude. If $D$ computed this way localizes failures equally well, then answer-relevance is not doing the work and the core claim fails.

Why not just…?

Why not just check whether the answer flips when the image is edited? This is the honest minimal baseline, and for the narrow question "is this correct answer grounded?" it may be nearly sufficient on synthetic charts. It cannot distinguish "dropped" from "misreasoned" when the answer is wrong. It also needs a controlled edit, which natural benchmarks do not provide. The experiments report flip rate beside the trace. If the trace adds nothing to flip rate on the taxonomy and detection tasks, we will report that result as the finding.

Why not just use GeoR or curvature on the single real-image trajectory? These measures capture instability, but shortcutting through a title is a stable failure. Plot Twist's title effect suggests many confident correct answers are ungrounded, and a stability score should rate these as healthy. The title-conflict variant is the direct test. If GeoR or curvature separates title-driven answers from bar-driven answers as well as $D$ does, the paired-trajectory idea is unnecessary.

Why not just convert the image to text and reason over that? GeoSQA found that models given expert text descriptions of diagrams still scored 23–26.22% against 25% chance. Table conversion as in Synthesize Step-by-Step depends on tabular structure that geometry diagrams lack. More fundamentally, conversion sidesteps the diagnostic question of what the end-to-end model does with what it sees; it does not answer it.

Why not just read attention to image tokens? Attention is cheap and it is a real threat. Attention mass shows where the model looks, not whether what it retrieved changes its internal state, and the comparison is included directly as a baseline. If attention mass predicts the dropped step as well as $D$, we will recommend it instead.

Decisive Experiments

On the title-conflict charts, the trace should show low $D$ for items answered with the title's value and high $D$ for items answered with the bars' value, with clearly separated distributions. If this fails, the measure does not capture grounding and the program stops.

Next, three annotators will label 200 MATH-Vision items with MATH-Vision's error categories and with the step at which visual information was dropped. The automatic taxonomy should agree with these labels better than the curvature and attention baselines do. We measure agreement with Cohen's kappa for categories and step-level accuracy within one step for location.

A causal check follows. Copying the edited image's visual-token states into the real-image run should change the answer much more often for items the trace marks "used" than for items it marks "dropped." Without this result the trace is only correlational.

Finally, the masked-image dependence score should add AUROC over GeoR for error detection on MATH-Vision and GAOKAO-MM under identical models and chains. For reference, MATH-Vision reports GPT-4o at 30.39% accuracy and GAOKAO-MM reports open-source models below 25%, so error-detection sets will contain plenty of errors.

A negative outcome would also be informative. If image-driven differences persist through the answer step in most wrong items, the perception story is largely ruled out for these models, and MATH-Vision's reasoning-error attribution would be confirmed mechanistically rather than by manual labels.