Grounded Langevin Refinement for Visual Math, Tested on Freshly Rendered Problems

When a vision-language model "thinks longer" about a chart or a geometry figure, it often gets worse at seeing it. As its written reasoning grows, its attention drifts from the image toward its own earlier text. It then answers from what it expects the picture to show rather than from what the picture does show. We propose a test-time procedure that refines a model's reasoning chain by noisy gradient steps. The objective rewards a correct answer, and it also rewards reasoning that would change if the image were different. We test this procedure only on chart and diagram problems rendered fresh from programs at evaluation time, so the model cannot have memorized them.

The two halves need each other. On static public benchmarks, ignoring the image can pay off, because the language prior may hold a memorized answer or a leaked chart title. A method that forces the model to keep looking can therefore look useless on contaminated test sets and still be valuable on new problems. Our central claim is falsifiable on two counts. First, adding an image-dependence term to continuous test-time optimization reduces the reasoning-versus-perception trade-off at long compute budgets. Second, its benefit is systematically underestimated on static benchmarks compared with matched, freshly generated twins.

Background and the gap

The failure mode is well documented. More Thinking, Less Seeing? shows across eight multimodal reasoning models that longer chains of thought amplify visual hallucination, and traces this to attention moving away from image tokens. It introduces RH-Bench and the RH-AUC metric, the area under the reasoning-versus-hallucination curve across token budgets, and reports that the reinforcement-learning-only Ocean-R1-7B retains grounding best, with RH-AUC 0.63. The paper diagnoses the problem but gives no mechanism that prevents drift during generation, so it motivates this work.

Two other papers point the same way. MATH-Vision finds that 42.2% of GPT-4V failures are reasoning errors made after correct perception, against 31.9% recognition errors. Plot Twist shows chart accuracy falling from 79% to roughly 30% when the answer is removed from the title. Models lean on text shortcuts, and static benchmarks reward them for it.

The method we borrow comes from ∇-Reasoner. It treats a reasoning chain as a relaxed, continuous sequence of token logits and improves it by gradient descent on a differentiable reward, with a penalty that keeps it close to the base model's distribution. It lifts Qwen-2.5-7B-Instruct on MATH-500 from 71.2% to 80.4% and beats Best-of-8 while using up to 40.2% fewer model calls. It is text-only and deterministic, it is evaluated on public competition sets, and nothing in its objective asks the chain to depend on an image.

Imagination Helps Visual Reasoning, But Not Yet in Latent Space uses causal mediation analysis on latent visual reasoning tokens. It finds that perturbing the input image barely changes those tokens, and that the tokens collapse to near-identical vectors across tasks. That is a constraint on our design: optimizing in a continuous space does not by itself keep reasoning tied to the input. We therefore make input dependence an explicit term in the objective rather than hoping it emerges. The opposing result in Reasoning in the Dark, which reports accuracy and speed gains from interleaved latent vision-text steps, does not measure input dependence. Our causal check below speaks directly to the disagreement.

On the evaluation side, DyVal generates reasoning problems from random graphs at controlled depth. It shows models that score well on static sets collapsing to near zero on fresh ones. A²RBench generates tasks whose validity is checked by running code that converts each input forward and back again. Both stay in text or discrete grids. We supply their missing visual counterpart: rendered charts and diagrams whose answers come from the generating program.

Core Claim

Let $z$ be the relaxed logits of a reasoning chain for question $q$ about image $I$. Let $\tilde I$ be a corrupted image, either blank or with a different random rendering of the same template. We define a grounding score

$$G(z) = \log p_\theta(z \mid I, q) - \log p_\theta(z \mid \tilde I, q),$$

where $p_\theta$ is the base model's likelihood. $G$ is high when the chain is far more plausible given the true image than given a wrong one, which is exactly the property Imagination Helps Visual Reasoning, But Not Yet in Latent Space found missing. We minimize the energy $U(z) = -R(z) + \lambda D(z) - \beta G(z)$. Here $R$ is a differentiable correctness reward, $D$ is ∇-Reasoner's divergence from the base model's distribution, and $\lambda, \beta \ge 0$ are weights. We minimize it with the overdamped Langevin update, which is gradient descent plus injected Gaussian noise:

$$z_{k+1} = z_k - \eta \nabla U(z_k) + \sqrt{2\eta T}\,\xi_k, \qquad \xi_k \sim \mathcal N(0, I).$$

Here $\eta$ is the step size and $T$ is a temperature that we anneal toward zero. Setting $T=0$ and $\beta=0$ recovers ∇-Reasoner. Setting $T=0$ with $\beta>0$ isolates the grounding term. Setting $T>0$ with $\beta=0$ isolates the noise.

We expect the grounding term to carry most of the benefit. The noise is a secondary hypothesis. Sampling texts such as Sampling Algorithms in Statistical Physics motivate noise as a way to escape poor local optima. But random restarts of deterministic descent do the same job cheaply, so we test the two against each other directly rather than assume the noise matters.

Research Program

We build three components.

The refinement procedure. We implement it for Qwen2.5-VL-7B and Ocean-R1-7B, both used in More Thinking, Less Seeing?. Gradients pass through the image-conditioned model with the same straight-through relaxation ∇-Reasoner uses. $G$ needs one extra forward pass per step, on $\tilde I$.

The reward $R$ is the weak point, because the file contains no differentiable multimodal reward model. We use the base model's own log-probability that its final answer is correct, and also report a text-only variant. During refinement we never use the program's ground truth.

The procedural benchmark. Parameterized programs render bar, line and scatter charts and coordinate-geometry figures. Each item is produced by a random seed, and its answer is computed by the program. Every question is executed and re-derived, in the spirit of A²RBench, before it is admitted.

Two difficulty axes are varied independently. Perception difficulty is controlled by marker density, gridlines, absent or misleading titles, and unlabeled values, following the probes in Plot Twist. Reasoning depth is the number of chained operations, from D1 to D4 as in DyVal. Items are regenerated with fresh seeds at each evaluation and never released, so they cannot be scraped.

We also build "static twins": items rendered to imitate the format of ChartQA and MathVista problems, for paired comparison with the published static sets.

The analysis. We trace accuracy, perception accuracy and RH-AUC as compute grows, measured in forward-pass equivalents, where one gradient step counts as roughly three forward passes. We also run a causal test of input dependence: after refinement, we swap the image and measure how much the chain and the answer change.

Baselines

Greedy chain-of-thought. The naive reference point is the base model generating a single chain of thought greedily, at matching token budgets. It tells us whether any test-time compute helps on fresh items. For context, the static MATH-Vision paper reports GPT-4o at 30.39% and human performance at 68.82%.

Best-of-N. The strongest discrete competitor samples $N$ chains at high temperature and selects by majority vote or by the same self-judged reward $R$. We set $N$ so its forward-pass equivalents match each Langevin budget, and give it the same temperature-tuning budget we spend on $\eta$, $T$ and $\beta$. There is no reported result on Lacuna for Best-of-N on MATH-Vision or ChartQA with these models, so we establish the number ourselves.

∇-Reasoner, ported to images. The strongest existing continuous method is ∇-Reasoner run on the vision-language model with $T=0$ and $\beta=0$. The published figure to hold in mind is MATH-500 accuracy improving from 71.2% to 80.4% on Qwen-2.5-7B-Instruct, a text model. There is no reported result on Lacuna for ∇-Reasoner on any multimodal benchmark. Our port therefore uses its published fluency weighting and step schedule, with the same tuning grid we allow ourselves.

Ablations. Two runs isolate the ingredients: grounding without noise ($T=0$, $\beta>0$) and noise without grounding ($T>0$, $\beta=0$).

What counts as winning. On fresh procedural items, the full method must beat both Best-of-N and ported ∇-Reasoner at matched compute, by more than the seed-to-seed variation across three regenerated test sets. It must also exceed the RH-AUC of 0.63 that More Thinking, Less Seeing? reports for Ocean-R1-7B on RH-Bench, when evaluated on RH-Bench itself.

Why not just…?

Why not just use Best-of-N with a verifier? It is simple, parallel and strong. But every sample is still an ungrounded long chain, and More Thinking, Less Seeing? shows that drift grows with chain length, so voting may simply pick the most confident hallucination. This is a genuine threat on easy-perception items. The matched-compute comparison, split by perception-difficulty level, will show whether Best-of-N falls behind only where seeing is hard, which is what we predict.

Why not just restart deterministic descent from several random points instead of adding noise? This is the alternative most likely to match our noise term, and we do not argue around it. The noise-only ablation runs against $K$ restarts of ∇-Reasoner at equal compute. If restarts match or beat annealed Langevin, we drop the noise. The contribution then reduces to grounded continuous refinement plus the contamination finding, and the paper will say so.

Why not just add a grounding penalty without any continuous optimization, for example by reranking sampled chains by $G$? This is cheap and could capture much of the effect. We include it as an extra baseline: Best-of-N reranked by $R + \beta G$. If it matches gradient-based grounding, the lesson is that the image-dependence signal matters and the optimizer does not, which is still worth knowing.

Why not just make the model describe the image in words before reasoning, as CapImagine does? Imagination Helps Visual Reasoning, But Not Yet in Latent Space shows that explicit descriptions beat collapsed latent tokens. However, its authors concede that words lose pixel-level detail. The scatter-plot coordinate items, where Plot Twist reports accuracies as low as 4.2%, test exactly that loss. We run a training-free "describe first" prompt as a further baseline at matched budget.

Why not just perturb static benchmarks, for example by removing titles, instead of generating new items? Perturbed items become scrapeable once published. Perturbation also cannot vary reasoning depth independently, and DyVal shows that depth is where models break. The static-twin comparison will measure how far static scores, perturbed or not, mislead.

Proposed Experiments

The first decisive check covers compute curves on fresh procedural items, crossed with perception difficulty and reasoning depth. Grounded refinement should flatten the decline in perception accuracy that appears at long budgets. That decline is the "more thinking, less seeing" curve, and we predict the largest gains in the hard-perception, deep-reasoning cell.

The second check tests the contamination prediction. We compare each method's gain over greedy decoding on the published static sets (MathVista, MATH-Vision, ChartQA) against its gain on their freshly rendered twins. The prediction is that the gain from grounding is larger on the fresh twins, while the gain from Best-of-N is similar on both. If gains are equal on both, the contamination half of the claim is wrong.

The third check is causal. After refinement, swapping the image should change the final answer far more often under grounded refinement than under ∇-Reasoner, which addresses the input-disconnect finding directly.

The fourth check is the restart-versus-noise comparison described above. We also report RH-AUC on RH-Bench, so our results connect to the one published grounding number.