Checking every step of a language model's reasoning, rather than only its final answer, catches errors that would otherwise compound. It is also slow. The most direct multi-judge approach to step checking runs several frozen judge models on every step and reconciles their scores as a game. VERDICT does exactly this, and it reports a 3.80x wall-clock overhead over the base model. This proposal asks a narrower and more useful question: which steps actually need the expensive multi-judge check? A cheap signal should handle the easy steps, and the panel should be called only when the evidence about a step is genuinely ambiguous.
The test for ambiguity comes from game theory. A Nash equilibrium is a set of independent strategies from which no player gains by deviating alone. Computing one in a general game is PPAD-complete, a complexity class believed to contain no efficient algorithm. A correlated equilibrium relaxes this. A neutral coordinator privately recommends an action to each player, and the recommendation must be one each player is willing to follow. The set of correlated equilibria is a convex polytope cut out by linear inequalities, so linear programming can optimize over it in polynomial time. We treat each step as a small game among heterogeneous judges and measure how much the step's acceptance probability can vary across every correlated equilibrium. If all equilibria agree, the step is settled cheaply. If the polytope spans both accept and reject, the incentives and evidence do not pin down an answer, and the step is escalated to more judges, resampled, or rolled back.
Core Claim
The accuracy of verifier-guided search is capped by false positives at high confidence. ROC-n-reroll proves that the test-time scaling ceiling is set by the slope of the verifier's ROC curve at the origin, which is the region where the verifier accepts almost nothing. Our claim has two parts.
First, a cascade can recover most of the low-false-positive behavior of a full judge panel at a fraction of its latency. In the cascade, a tiny internal-state probe scores every step, and a step reaches the panel only when the probe score falls in an uncertain band and the correlated-equilibrium width is large.
Second, the equilibrium width is a better escalation trigger than raw score dispersion, but only when the judges are heterogeneous. A heterogeneous panel might combine a strict symbolic checker that is known to over-penalize informal steps, a lenient contextual judge, and a learned scorer. In that setting the judges' payoffs differ in how much they lose from a false accept versus a false reject. Raw score spread ignores those asymmetric costs.
We state the second part as a hypothesis that can fail. If plain dispersion gating matches width gating at the same escalation rate, the game-theoretic layer is unnecessary, and we will report that result.
Why per-step games, not one game over the whole chain
The obvious way to join these directions is to model the entire chain of thought as an extensive-form game, meaning a game tree in which earlier moves shape later options. One would then compute extensive-form correlated equilibria with uncoupled no-regret learning. No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium supplies that method. Its algorithm (ICFR) splits regret into local problems at each decision point and solves Leduc Poker in about 9 hours. It still keeps an explicit regret minimizer at every decision point, and that paper itself notes that choosing the best equilibrium in extensive form is hard.
That cost profile is incompatible with a latency budget measured in multiples of a single forward pass. We therefore treat the complexity results as a design constraint rather than a mechanism to exploit. Each step gets its own tiny normal-form game: a one-shot game with $k \le 4$ judges, two verdicts each, and $2^k$ joint outcomes. The dependence between steps is handled by the search procedure, not by the game.
This choice also separates the proposal from VERDICT. VERDICT designs a quadratic coordination game so that its unique Nash equilibrium has a closed form. Those clean solutions disappear once judges have asymmetric, non-quadratic payoffs, and we drop that restriction. VERDICT then thresholds a single equilibrium point. We instead use the extent of the whole equilibrium set as the signal.
Research Program
In each step game, judge $i$ chooses a verdict $v_i \in \{\text{accept}, \text{reject}\}$. Its payoff combines three terms: fidelity to its private score $s_i$ for the step, a weight $\lambda$ on agreeing with the other judges, and asymmetric costs for false accepts and false rejects, set per judge. For example, the symbolic unit checker pays heavily for a false accept and the contextual judge does not. Let $\mathcal{C}$ be the polytope of correlated equilibria, defined by the standard no-profitable-deviation linear constraints. The escalation signal is
$$ W = \max_{\pi \in \mathcal{C}} P_\pi(\text{majority accepts}) - \min_{\pi \in \mathcal{C}} P_\pi(\text{majority accepts}), $$
where $\pi$ is a joint distribution over verdict profiles. $W$ takes two linear programs with at most 16 variables to compute, so its cost is negligible next to model inference.
One risk is clear in advance. If $\lambda$ is large, both "everyone accepts" and "everyone rejects" become equilibria regardless of the evidence, and $W$ saturates near 1. To prevent this, $\lambda$ and the cost weights are fitted on a held-out split to maximize the ROC slope near the origin, and we report how sensitive $W$ is to $\lambda$.
The cascade has three tiers, in the spirit of the adaptive verification VERDICT lists as future work.
The first tier is a ReProbe-style probe. Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models trains a probe of under 10 million parameters on the generator's hidden states, reports 2.6x to 25x speedups over generative reward models, and matches models up to 810x larger on PR-AUC. We reuse that method as tier one, scoring every step.
The second tier is a cheap symbolic or unit checker when the domain has one. It is plugged in as one judge in the game, so the game itself can discount its known strictness.
The third tier is the full LLM judge panel, which is called only when $W$ exceeds a threshold.
These tiers sit inside step-level rejection sampling. A candidate next step is accepted once the cascade accepts it robustly. Otherwise it is resampled up to a cap, and repeated failure triggers a rollback to the last accepted step, following Thought Rollback. This extends the solution-level analysis in ROC-n-reroll to verification during generation. That analysis is the source of our prediction that rejection sampling should beat best-of-$N$ on compute at equal accuracy whenever the composite verifier's ROC curve is concave.
Baselines
The naive reference is the base generator decoding greedily with no verification. Next to it we run outcome-level best-of-$N$ and capped rejection sampling with the probe as the final-answer verifier. ROC-n-reroll establishes rejection sampling as the more compute-efficient of the two, but it gives no numeric GSM8K or MATH500 accuracy; no reported result was found on Lacuna for those values.
The strongest single-verifier competitor is probe-only step-level rejection sampling. It runs the same probe used as our tier one with no escalation. It is the strongest competitor on latency because it adds almost no cost. No reported result was found on Lacuna for its benchmark accuracy on MATH or GSM8K beyond the stated speedups and the PR-AUC parity claim. To beat it, our cascade must reach higher final accuracy and a steeper low-false-positive ROC region while staying within a stated wall-clock multiple of it.
The strongest multi-judge competitor is VERDICT, which runs the full Nash-consensus panel on every step. VERDICT reports a +5.95% accuracy gain on CV-Bench-3D, gains of 2.4% to 5.2% across its six benchmarks, and 3.80x sequential overhead, which it projects could drop to 1.59x with parallel execution. Beating it means matching its accuracy within noise on CV-Bench-3D and the math benchmarks at lower wall-clock time than its parallel 1.59x figure. Running all judges on every step with score averaging serves as the simple panel reference.
The ablation that isolates the key ingredient is an identical cascade gated by raw score dispersion instead of $W$. A second variant is gated by VERDICT's equilibrium dispersion. Both are compared at matched escalation rates.
Every method uses the same generator, judges, probe checkpoint, sampling cap, and held-out tuning split. Every method receives the same threshold-search budget.
Why not just…?
Why not just average the judges' scores or take a majority vote? VERDICT reports that averaging "buries" conflicts. Scores of (0.9, 0.3, 0.9) and (0.7, 0.7, 0.7) average to similar values but mean very different things. VERDICT's equilibrium consensus beat averaging, and the margin widened with 2B-parameter judges. We include averaging both as a baseline and as an escalation signal. Averaging also cannot reduce latency on its own, because it still calls every judge on every step.
Why not just use the probe alone? This is the most serious threat on any single-model benchmark. The probe is cheap and strong, and if its errors are mostly confident false positives, escalation will rarely trigger and will add little. However, ReProbe probes are tied to one model's hidden states, so they cannot cover API-only generators. Our ROC-near-origin measurement will show directly whether escalated judges remove the probe's high-confidence false positives. If they do not, the probe alone wins, and we will say so.
Why not just gate on score dispersion, without any game? This is the alternative most likely to work as well as our method. We expect $W$ to help only when judges have asymmetric costs, for instance a strict formal checker of the kind Efficient PRM Training Data Synthesis via Formal Verification shows over-penalizes informal steps. The matched-escalation-rate ablation settles the question. We run it on a homogeneous panel, where we predict no difference, and on a heterogeneous panel, where we predict an advantage for $W$.
Why not just train a dedicated critic? VERDICT reports that trained critics such as LLaVA-Critic and Sherlock varied by task and sometimes fell below baseline, while training-free consensus stayed positive on all six benchmarks. A trained critic is also a single verifier, so it inherits the same ROC ceiling. We therefore treat a trained critic as a candidate for tier three, not as a substitute for the cascade.
Proposed Experiments
The primary setting is math reasoning on GSM8K, MATH500, and ProofNet. Step correctness labels come from PRM800K, which is available through the VersaPRM resource and is the data ReProbe was trained on. FOVER-40K labels supply the strict formal judge. BioDimBench tests a heterogeneous panel with a unit checker. Step-aware checking already reaches 1.000 F1 there, so it serves as a check on latency and robustness rather than an accuracy contest. CV-Bench-3D, BLINK, and MMStar give a direct comparison with VERDICT.
The first decisive check is the empirical ROC slope near the origin for the probe alone, for each judge, for the full panel, and for the cascade. The question is whether escalation raises that slope, which is the open question ROC-n-reroll leaves for multi-verifier search.
The second check is the full Pareto curve of final accuracy against wall-clock time and judge calls, with VERDICT's 3.80x and 1.59x figures marked.
The third check is the width-versus-dispersion ablation on homogeneous and heterogeneous panels.
The fourth check is a stress test for correlated failure. Judges that share one backbone should produce narrow equilibrium sets that are wrong, the "confidently wrong" case flagged against VERDICT. We measure how much of the gain survives when judge diversity is removed.
The proposal is worth pursuing if the cascade stays within a small multiple of probe-only latency while matching panel-level accuracy, and if width gating beats dispersion gating specifically on heterogeneous panels.
References
- VERDICT: Training-Free Step-Wise Verification of Multimodal Reasoning via Disagreement-Aware Consensus
- ROC-n-reroll: How verifier imperfection affects test-time scaling
- No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
- Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models
- Toward Adaptive Reasoning in Large Language Models with Thought Rollback
- Efficient PRM Training Data Synthesis via Formal Verification
- BioDimBench: Verifying Unit-Consistent Biomedical Mathematical Reasoning