Premise Audits: Making Verifier-Guided Search Robust to Distracting Context

When a language model is given more thinking time, it sometimes gets worse. This happens especially when the question contains irrelevant or misleading text, because the model spends the extra time weaving that text into its answer. Yet one controlled study found the opposite result for a different way of spending compute: searching over many candidate solutions and letting a separate scoring model (a verifier) pick the best partial steps made models more robust to irrelevant context. We propose that the verifier decides which result you get. Extra compute filters out distractors only if the verifier actually checks whether each reasoning step relies on legitimate parts of the question. If it does not check this, a larger search budget simply finds more fluent rationalizations of the distractor.

Our method is a premise audit. Before search begins, a model reads the problem and writes a short rubric specific to that problem, meaning a list of checkable criteria. The rubric marks each input clause as needed, irrelevant, or suspect, and states what a step may and may not derive from it. During search, every candidate step is scored against this rubric in addition to the usual correctness check. The new element is not rubric-based rewards or search as such. It is moving the question "which premises is this step allowed to use?" into the verifier, where it can be applied softly, step by step, and differently across search branches. It does not belong in an upstream filter that commits once. We test whether this turns inverse scaling under distraction back into positive scaling, and whether it beats the simpler fixes a reviewer would suggest first.

Core Claim

The direction of test-time scaling under contextual noise is set by one property of the verifier: how much its score for a step changes when that step consumes a distractor. Current process verifiers score each intermediate step, and they are largely insensitive to this. A verifier that audits which premises each step uses will produce accuracy that rises with search budget on distracted inputs. The same search with a standard process verifier will plateau or decline at high budgets. We define the robustness gap at compute budget $B$ as

$$\Delta(B) = \mathrm{Acc}_{\text{clean}}(B) - \mathrm{Acc}_{\text{distracted}}(B),$$

where both accuracies are measured on matched clean and distracted versions of the same problems. The claim is that $\Delta(B)$ shrinks as $B$ grows under premise-audited search, and stays flat or grows under unaudited search and under simple lengthening of the chain of thought.

What Prior Work Establishes and Leaves Open

Several papers motivate the problem by showing that extra reasoning amplifies distraction. Inverse Scaling in Test-Time Compute shows that longer reasoning traces lower accuracy on counting tasks padded with misleading math and Python snippets, and on constraint puzzles. It only studies self-directed reasoning length, not verifier-guided search. Thinking in a Crowd identifies a "rationalization trap." With misleading snippets, Qwen3-8B falls from 96.22% to 77.03% with thinking on. That is barely above thinking off at 74.87%, and the traces are longest precisely when rationalizing a false premise. It compares only thinking on and off. GSM-Symbolic shows that one irrelevant clause (GSM-NoOp) cuts o1-preview accuracy by 17.5% and Phi-3-mini by 65%, because models turn the clause into arithmetic.

The counterpoint comes from How Is LLM Reasoning Distracted by Irrelevant Context?. It builds the GSM-DC benchmark, in which math problems are generated as graphs of quantities and distractors are nodes off the solution path. It finds that search over a tree of partial solutions, guided by a process reward model (a verifier that scores each intermediate step), improves robustness. It uses a fixed, off-the-shelf process reward model and never asks whether that model itself is fooled by distractors. Our hypothesis is designed to reconcile it with the inverse-scaling results.

A second group of papers constrains the design by showing that current verifiers are the weak link. Reward Under Attack finds that Qwen2.5-Math-PRM-7B barely changes its scores when the question is removed entirely (peak change near 0). It also finds that Skywork's model largely fails to penalize hallucinated reasoning, with a score change centered near 0. Both reward fluency over validity. A verifier that ignores the question cannot notice that a step used an irrelevant clause from it. Scaling Test-Time Compute Without Verification or RL is Suboptimal proves that verifier-guided search beats verifier-free scaling. It also reports that imperfect verifiers get exploited at large budgets, which is exactly the regime where we expect distracted search to fail. It leaves process-level rewards as an open extension, and our work supplies one.

A third group supplies the machinery we reuse. GenPRM turns the process verifier into a model that writes a critique and runs verification code for each step, and it scales the verifier's own compute by majority vote. This yields 80.5 F1 on ProcessBench against 78.3 for Qwen2.5-Math-PRM-72B. It is evaluated only on clean math, and its own chain-of-thought verification may fall into the same rationalization trap. Step-wise Rubric Rewards and Curing Miracle Steps with Rubric Rewards show that rubrics attached to individual steps catch unsound reasoning that outcome rewards miss. The second paper reduces unjustified leaps ("miracle steps") by 71%. Both build rubrics offline, apply them during training rather than as a search-time signal, and test only on clean problems. CARMO and Thinking-LLM-as-a-Judge generate criteria per prompt at runtime, but for scoring whole responses, not individual steps, and without a notion of premise status. Dynamic and Generalizable Process Reward Modeling assigns criteria dynamically to steps from a pre-mined tree of criteria. Its criteria concern step quality, not whether an input clause is legitimate, and it is untested under noise. Premise audits combine per-problem criteria generation with step-level scoring. They are new in making the criteria about the input's clauses rather than about generic step quality.

Research Program

The first component is a measurement: a counterfactual test of whether a verifier notices distractor use. GSM-DC lets us label, with no human annotation, whether any step uses a quantity from an off-path node, because the generating graph is known. This matters because ProcessBench had to discard 30% of its expert labels for lack of consensus. We construct pairs of steps that are identical except that one pulls a number from a distractor node. For each verifier we then measure the score difference between the two steps, which we call distractor sensitivity.

GSM-NoOp gives the same kind of pairs at the level of single irrelevant clauses. For misleading rather than irrelevant context we use SciAux. There, a step that adopts a claim from a misleading snippet can be identified by matching it against the planted false claim. The first hypothesis to test is that distractor sensitivity on these pairs predicts the sign of $d\Delta/dB$ across verifiers. If it does, that alone explains the conflict between the inverse-scaling results and the GSM-DC result.

The second component is the premise audit itself. A rubric writer is given the problem and splits it into clauses. For each clause it writes a status (needed, irrelevant, or suspect) and a checkable criterion, for example "no step may use the 5 kg weight of the basket" or "the snippet's claim that X causes Y must not be adopted without support from the question." The verifier then scores each step on two things combined: its usual judgment of correctness, and compliance with the criteria for whichever clauses the step draws on.

Unlike an upstream filter, the full context stays visible to the solver, and the audit only down-weights steps. If the rubric wrongly labels a needed clause as irrelevant, branches that use it are penalized but not deleted, so a strong correctness signal can still carry them through. We also sample several rubrics and average their scores, which spreads the risk from any single wrong relevance call.

To avoid the teacher-model confound that the rubric-reward paper acknowledges, the rubric writer is the same model as the generative verifier, and its tokens count against the compute budget.

The third component tests scaling under distraction. We run Best-of-$N$ selection and step-level tree search over budgets from $N=1$ to $N=64$ and plot $\Delta(B)$ against total tokens, including the tokens spent writing rubrics and verifying. Separately, we scale the verifier's own votes, following GenPRM, from 1 to 8 on misleading SciAux inputs. This asks whether verifier-side reasoning shows its own rationalization trap. If it does, audits should help most exactly where plain generative verification gets worse with more votes.

Baselines

The naive reference point is the policy model itself, Qwen3-8B chosen to match the SciAux setup, with thinking on and then off, and with reasoning length raised by prompt forcing as in Inverse Scaling in Test-Time Compute. This is the "just think longer" curve. The reported reference numbers are Qwen3-8B on SciAux with two misleading snippets, at 77.03% with thinking on, 74.87% with thinking off, and 96.22% clean (Thinking in a Crowd).

Majority voting over the same $N$ samples is the verifier-free baseline. It is cheap, and it is exactly the comparison the verifier-free scaling analysis says should lose.

The strongest existing method is step-level tree search guided by an unaudited verifier. We run it with two verifiers. The first is Qwen2.5-Math-PRM-7B, which reproduces the GSM-DC mitigation. The second is GenPRM-7B with 8-way voting, the strongest reported process verifier here, at 80.5 F1 on ProcessBench (GenPRM). We found no result on Lacuna for any verifier-guided search accuracy on GSM-NoOp or SciAux. For GSM-DC, the file reports that tree search guided by a process reward model improved robustness, but gives no accuracy figure. The only GSM-DC number available is the unguided collapse of Grok-3-Beta's step accuracy on 5-step problems, from 43% with one distractor to 19% with fifteen (How Is LLM Reasoning Distracted). We will therefore reproduce these baselines ourselves at matched budgets rather than compare against published figures.

Fairness is enforced by giving every method the same policy samples and the same token budget, with rubric-writing and verifier tokens counted. GenPRM receives the extra votes that our audit's tokens would otherwise buy. Each method gets the same hyperparameter sweep on a held-out split of distraction levels 1 to 5, and is evaluated on levels 6 to 15.

Two further baselines are the simple fixes discussed below. One is extract-then-solve, which selects the relevant chunks and then answers, in the style of Ext2Gen; we also run a version that votes over several extractions. The other is a policy fine-tuned on the hardest distractor settings (the Hard-IC recipe) and then decoded with majority voting.

The ablations isolate the premise audit. The first replaces the per-problem rubric with a fixed generic criterion ("the step uses only information relevant to the question"), which separates problem-specific criteria from simply mentioning relevance. The second keeps the rubric but hides the question from the verifier. The third drops the soft penalty in favor of hard pruning of flagged branches, which turns the audit back into a filter.

Beating the strongest baseline means a smaller $\Delta(B)$ at the largest budget on GSM-DC levels 10 to 15 and on SciAux misleading inputs, with confidence intervals that do not overlap across three seeds. It also means a positive slope of distracted accuracy with respect to $B$ where the unaudited verifier's slope is zero or negative. Finally, clean accuracy must stay within one point of the baseline on GSM8K and MATH500, so that we have not simply bought robustness at the cost of clean performance.

Why not just…?

Why not just tell the model to ignore irrelevant information? Instructions do not reliably stop text models from converting irrelevant clauses into arithmetic. GSM-NoOp's drops of 17.5% for o1-preview and 65% for Phi-3-mini persist in capable reasoning models (GSM-Symbolic). The foreground-focused prompt that worked in the vision-language distractor study addressed crowding by visual attributes, a different failure. We still include an "ignore irrelevant information" system prompt on every method, so any gain from auditing is measured on top of it.

Why not just filter the context before solving? This is the most serious alternative, and on GSM-DC's purely irrelevant distractors it may match us. Relevance there follows from the graph structure, and a good extractor might recover it. We expect filtering to fall short in two places. The first is misleading context, as in SciAux and the misleading-math tasks, where the harmful snippet is on topic and a relevance filter keeps it. There, the audit's "suspect" status, which checks the claim rather than its topic, is what matters. The second is deeper problems, where a filter that wrongly drops one needed premise makes the problem unsolvable at any budget. The hard-pruning ablation measures exactly this cost of committing. If extract-then-solve with voting matches premise audits on SciAux misleading inputs at equal tokens, we will conclude the audit adds nothing beyond filtering and report it as such.

Why not just use a stronger verifier or more verifier votes? GenPRM shows that scaling verifier compute raises clean ProcessBench F1. However, Reward Under Attack shows that strong process verifiers can ignore the question altogether, and more votes from a question-blind verifier should not fix a question-level error. Our verifier-vote sweep on SciAux tests this directly. If GenPRM's distractor sensitivity rises with votes, and its search curve matches ours at equal tokens, then auditing is unnecessary.

Why not just train the solver on hard distractors? Hard-IC training generalizes across distraction levels (How Is LLM Reasoning Distracted), but it requires retraining for each distractor type. We test whether a Hard-IC-trained solver transfers from GSM-DC's irrelevant nodes to SciAux's misleading snippets. We also test whether audits and Hard-IC training add up when combined, which would make them complementary rather than competing.

Decisive Checks

The first check is the correlation. Across the four verifier variants (standard process reward model, GenPRM, generic-criterion audit, premise audit), distractor sensitivity measured on GSM-DC counterfactual pairs should rank-order the verifiers' $d\Delta/dB$ on held-out GSM-NoOp and SciAux. If it does not, the core claim is wrong, whatever the method's headline accuracy.

The second check is the sign flip. On the misleading-math task in the inverse-scaling codebase and on SciAux misleading inputs, unaudited verifier-guided search should show flat or declining accuracy as $N$ grows from 16 to 64, while premise-audited search keeps improving.

The third check is the process metric. On GSM-DC, the fraction of selected final trajectories that use any distractor node should fall with budget under auditing and rise or stay flat without it. Because this is measured from the generating graph, it confirms that accuracy gains come from avoiding distractors rather than from luck in final answers.

The fourth check is the clean control. On GSM8K and MATH500, auditing must not cost more than one point, confirming that the method removes a specific failure rather than trading one error for another.