When a vision-language model solves a chart or geometry problem step by step, it often fails at a small perceptual claim rather than at the logic: it misreads an axis label or a coordinate. Search methods that branch over candidate steps and keep the good ones should catch these mistakes. In practice they rely on the model grading its own steps, and that grading is unreliable. The grader reads the same reasoning chain and the same distracting context that caused the error, so it tends to argue that the error is correct. We propose a narrower check. Each reasoning step must state its visual evidence as an explicit claim: this region of the image shows this value. Each claim is then checked by a fresh query that sees only a crop of that region and the claim, with no reasoning chain and no surrounding prompt. Many branches of a search tree repeat the same visual claims, so the answers can be cached. Checking then costs roughly once per distinct claim instead of once per branch.
The two-sentence version is this. Self-verification in multimodal reasoning fails largely because the verifier is conditioned on the chain and context it is meant to audit. Stripping that context, and verifying only atomic visual claims on image crops, should give a step checker that is both more accurate and cheap enough to guide inference-time search. The proposal treats the cross-modal perception gap as the source of errors, self-correction failure as the mechanism to fix, contextual noise as the stress test, and search overhead as the budget constraint.
Core Claim
Our hypothesis concerns the verifier, not the search algorithm. Two published results frame the problem.
When is Tree Search Useful for LLM Planning? studies text and SQL planning. It finds that tree search beats simply re-ranking complete samples only when the discriminator (the model that scores candidates) is roughly 90% accurate, and that below that level search runs 10 to 20 times slower for no gain. On Spider, iterative self-correction dropped CodeLlama-13B from 61.5% under re-ranking to 51.7%. This constrains us: a new search procedure is pointless unless the step checker is much better.
Thinking in a Crowd motivates our diagnosis. It shows that explicit reasoning over misleading snippets pulls Qwen3-8B from 96.22% to 77.03% on SciAux while responses grow longer as the model rationalizes the contradiction. On the self-verification limitations of LLMs reports related failures. GPT-4's self-critique rejected valid Mystery Blocksworld plans with a 97.09% false-negative rate, meaning it rejected almost every valid plan. By contrast, independent sampling with a sound external checker did far better, for example 44% versus 2% on Graph Coloring. AlphaGRPO finds the multimodal analogue, confirmation bias toward the model's own outputs.
Our reading of this evidence has two parts. First, verification is not inherently easier than generation, which is the lesson of Valmeekam et al. Second, it can be made easier by changing its input: remove the chain, remove the distracting context, and shrink the question to reading one region. A crop query of the form "does this region show 40?" does not depend on the long context that attention spreads across. We claim that this context-blind check reaches much higher accuracy on perceptual steps than chain-conditioned self-grading, and that it degrades far less as distractors are added.
Research Program
We will build three components on an open vision-language backbone, Qwen2-VL-7B. We choose it so that we can compare directly with Mulberry, which uses the same base model.
Claim-emitting reasoner. The first component makes the reasoner produce checkable claims. It is a lightly fine-tuned or prompted policy whose steps attach structured claims: a bounding box, an attribute (text, value, count, relation), and a stated value. Training data comes from Mulberry-260k trajectories HJYao00/Mulberry, with boxes added automatically. Steps with no visual content, such as pure arithmetic, are sent to Python execution when possible and are otherwise left unchecked. We report how often this happens, so the scope of the method stays visible.
Claim checker. The second component is the checker itself. It is the same backbone, or optionally a smaller one, queried with only the enlarged crop and a question derived from the claim. It returns a calibrated probability that the claim holds. This borrows from OmniVerifier-M1, which replaces verbal critiques with symbolic region-level rationales scored by rules and reports 0.021ms versus about 20ms per reward. Our setting differs in one important way. At test time there is no reference box to compute overlap against, so our check is a re-perception query rather than a rule. It is also applied to intermediate reasoning claims, whereas OmniVerifier-M1 applies to outcome and editing verification.
Memoized step-level search. The third component runs beam search or a small tree search over steps. A step's score is the product of its claims' check probabilities, and a step is pruned when any claim falls below a threshold. Checks are cached by claim, keyed on quantized box and attribute, and shared across branches.
The cost model is simple. Total cost is roughly generation cost plus $U \cdot c$, where $U$ is the number of distinct claims across the tree and $c$ is the cost of one crop query. Sibling branches over the same figure reuse regions, so we expect $U$ to grow sublinearly in the number of branches. This is an empirical prediction, and we will measure $U$ directly.
This design also responds to the draft-then-verify scheme of SpecSearch, where thought generation took over 91% of runtime. SpecSearch reduces that cost with a small drafter, but it inherits the weakness it names itself: a reward model fooled by plausible steps. Our checker targets exactly the plausible-but-misperceived step.
Noise stress sets. Finally, we build the noise conditions. We will construct multimodal distraction sets in two forms. The first follows SciAux billhdzhao/SciAux, which is derived from ScienceQA. We restore the ScienceQA images and prepend its misleading and irrelevant snippets. The second adds visual clutter to MathVista and VeriSciQA datajuicer/VeriSciQA figures: distractor panels, extra legends, and near-duplicate curves. We also add an adaptive variant that uses the tree-search distractor generation of Adaptive Distraction, which reported roughly 45% average drops on text benchmarks. We retarget it to maximize our system's error, so that robustness is not measured only against fixed noise.
Baselines
Plain chain-of-thought. The naive reference is the backbone's single-pass reasoning, run with the same prompt format and no search. It fixes the floor on every clean and noisy split and tells us whether search helps at all.
Best-of-N re-ranking. The strongest competitor for the search component is re-ranking N complete samples. Chen et al. identify it as the right choice whenever the discriminator is below about 90%. We run it with three different scorers: chain-conditioned self-grading, a multi-model voting committee, and our own claim checker. All runs share the same backbone and the same total budget, counted as matched backbone forward passes and wall-clock time on identical hardware. N is tuned on a held-out split with the same number of configurations as our beam width and threshold.
Mulberry-7B. The strongest existing multimodal system is Mulberry-7B, a Qwen2-VL-7B model fine-tuned on collective tree-search trajectories. Its reported MathVista score is 63.1%, against 58.2% for the Qwen2-VL-7B base (Mulberry). We run the released checkpoint both with and without our search wrapper. Beating it means exceeding its clean MathVista accuracy at matched compute and, more importantly, losing substantially less accuracy under our distraction splits.
VisuoThink. On Geomverse-109, the file reports only a relative figure for VisuoThink: a 17.1% gain over chain-of-thought, with no absolute accuracy. We rerun VisuoThink's self-voting search on our backbone. On the noisy VeriSciQA and multimodal SciAux splits, we found no reported result on Lacuna for any method, so those baselines are established by our own runs.
Context ablations. The ablation that isolates our key ingredient varies what the checker sees while holding the claim format fixed. It runs in three conditions: the full chain plus image, the full image plus the claim with no chain, and the crop plus the claim, which is our method. We also run the search with the cache disabled, which isolates the overhead contribution.
Why not just…?
Why not just use best-of-N with a good checker instead of search? This is a real threat, and we treat it as a possible outcome rather than a defeat. If the claim checker is the source of the gains, it should also improve re-ranking. Chen et al. predict that search pulls ahead only when step accuracy is high enough, and that complete-sample re-ranking weakens only on long compositions where full rollouts are rarely correct. We will measure the checker's step accuracy against the roughly 90% mark and report the crossover as a function of problem length. If re-ranking with our checker matches our search at equal compute everywhere, the contribution reduces to the checker, and we will say so.
Why not just perceive first and then reason in text? In this alternative, the model extracts all visual facts up front and then reasons purely over text. The approach removes the chain from perception, which is what we want, but the extraction cannot know in advance which facts the reasoning will need. It also passes along an unchecked, lossy description. GeoX found that natural-language descriptions of diagrams are too ambiguous, and it gained by aligning diagrams to formal strings instead. We will run a perceive-then-reason baseline at matched compute. A decisive sign for our approach would be that our method's advantage concentrates on questions whose needed facts only become clear mid-reasoning.
Why not just distill search offline, as Mulberry does? Distillation removes test-time cost. However, Mulberry reports that its models still reach correct answers through hallucinated intermediate logic, and a fixed checkpoint cannot adapt to distractors it never saw in training. Our adaptive-distraction split is designed to expose this. If Mulberry-7B degrades no more than our method under adaptive noise, the case for test-time checking collapses on that axis.
Why not just tell the model to ignore irrelevant context? Adaptive Distraction reports that such instructions do not fix distraction. Thinking in a Crowd shows that explicit reasoning can override them. We include an "ignore irrelevant information, answer unanswerable if unsure" prompt, in the style of DAS, as a cheap baseline.
Decisive Experiments
Step-level accuracy. The first check measures the checker directly, independent of any search. We take about 1,000 intermediate steps from MathVista and VeriSciQA trajectories and hand-label each visual claim, supplemented by the failed sibling branches in Mulberry-260k. We then compare the three context ablations on precision, recall and false-negative rate, on clean inputs and under 0, 1, 2 and 4 distractors. The hypothesis fails if crop-only checking does not beat chain-conditioned grading by a clear margin, or if its accuracy falls with the distractor count as steeply as self-grading does.
End-to-end accuracy and cost. The second check reports MathVista, Geomverse-109, VeriSciQA and multimodal SciAux accuracy as a function of backbone forward passes and wall-clock time, against every baseline above. It also records the number of distinct claims as tree width grows, which tests the sublinear cost prediction.
Width scaling. The third check targets the inverted-U failure that VisuoThink reports, in which widening the tree past a point lowers accuracy. We sweep tree width under noise. If the failure comes from evaluator error, the curve should flatten when crop checking replaces self-voting, and it should not flatten when checking is chain-conditioned.
Rationalization length. Finally, we track response length under misleading context. If pruning removes the rationalizing branches that Thinking in a Crowd documents, length growth under misleading snippets should shrink along with the accuracy loss.
References
- When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
- Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning
- On the self-verification limitations of large language models on reasoning and planning tasks
- AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in Unified Multimodal Models via Decompositional Verifiable Reward
- Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
- Multimodal Meta-Verifier with Explicit Structured Recalibration
- Accelerating Large Language Model Reasoning via Speculative Search
- Breaking Focus: Contextual Distraction Curse in Large Language Models
- VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search
- GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering