Many language-model systems improve an answer by looping. The model drafts a solution, critiques it, revises it, and repeats for a fixed number of rounds. Each round costs a full generation. Across vision, parsing, text generation and LLM reasoning, most of the gain arrives in the first one or two rounds. Later rounds are often worse than useless, because the model can "correct" a right answer into a wrong one. We propose a stopping rule for these loops that decides, after each round, whether to stop and which round's answer to return. Its thresholds are set on a small calibration set so that, with high probability, accuracy lost relative to running the full loop stays below a user-chosen budget $\varepsilon$.
The design borrows from database search. Progressive nearest-neighbour search finds the true answer early and then spends most of its time proving it. That work showed that a few hundred calibration queries suffice to predict, from the "best-so-far" trajectory, when further search is unlikely to help, and to stop with stated confidence (ProS; Progressive Similarity Search on Time Series Data). We apply the same idea to a refinement loop, with one change the search setting never needed. In search, extra work can only find a better candidate. In self-refinement, the current candidate can get worse. Our rule therefore has to guard against two errors: stopping too early, and continuing into a regression.
Core Claim
Sequential self-correction is the "depth" axis of test-time computation, as opposed to the "breadth" axis of sampling many independent answers. On this axis, a cheap trajectory-based stopping rule with finite-sample risk control can recover most of the accuracy of a fixed-depth loop at a fraction of its cost. Because the rule can also revert to an earlier round's answer, it can sometimes exceed fixed-depth accuracy.
The rule has three parts. It reads only signals the loop already produces: whether the extracted answer changed between rounds, the edit distance between consecutive drafts, how the length-normalized log-probability of each draft is trending, and whether the critique flagged an error. A small calibrated model maps these signals to two probabilities after round $t$. The first, $p_{\text{gain}}(t)$, is the probability that continuing will turn a currently wrong answer into a right one. The second, $p_{\text{reg}}(t)$, is the probability that the most recent change was a regression. The loop stops at the first round where $p_{\text{gain}}(t) < \lambda$, and returns the previous answer if $p_{\text{reg}}(t) > \lambda'$.
The thresholds $\lambda$ and $\lambda'$ are chosen on calibration problems by a standard distribution-free risk-control procedure. That procedure tests a grid of threshold pairs and keeps only those whose empirical risk is certified at the target level. The risk being controlled is
$$ R(\lambda,\lambda') = \Pr\big[\text{full-depth answer correct} \wedge \text{returned answer incorrect}\big] \le \varepsilon \quad \text{with probability at least } 1-\delta, $$
Here "full-depth" means running the loop for its fixed maximum depth $D$, $\varepsilon$ is the accuracy we are willing to lose, and $\delta$ is the failure probability over the draw of the calibration set. Among certified pairs, we pick the one with the lowest mean number of rounds. The guarantee assumes that calibration and test problems are exchangeable. Testing how the rule behaves when that assumption breaks is part of the program.
Relation to Prior Work
Four recent papers stop LLM computation early. None of them stops a sequential critique-and-revise loop with a guarantee, and each motivates or constrains our design differently.
CGES stops self-consistency sampling once a Bayesian posterior over answers concentrates. It works purely on the breadth axis, where samples are independent. That independence is exactly what fails in a refinement loop, where round $t$ conditions on round $t-1$, so its posterior update cannot be reused as is.
Statistical Early Stopping for Reasoning Models uses conformal prediction to bound false halting at $\alpha = 5\%$. Its stopping rule is a threshold on the density of uncertainty keywords, and its purpose is to abort ill-posed queries, not to detect that a solvable problem has converged. We reuse its methodological stance, finite-sample control of a halting error. We also heed its negative finding that trace length is a poor stopping signal, which is why our features avoid it.
Answer Convergence as a Signal for Early Stopping in Reasoning shows that answers stabilize long before a chain of thought ends. It stops on repeated answers or with a supervised classifier, but offers no error bound. It also warns that models can settle on wrong attractor states, a warning our regression-and-revert component takes seriously.
Multi-pass Decoding for GEC is the closest mechanism. It is a logistic classifier on the end-of-sentence hidden state that predicts whether another correction pass helps. That classifier is uncalibrated, binary, specific to encoder-decoder models, and cannot revert.
Three further lines of work shape the design. On the refinement side, Refining Over Resampling supplies our testbed. It shows that breadth-plus-depth self-refinement beats verifier-based Best-of-N by large margins, but it uses a hard-coded depth and acknowledges both cost and regression. Two works constrain how stopping should be done. When Denoising Hurts finds that late iterations of reverse diffusion actively degrade forecasts, yet it stops at one global breakpoint for the whole dataset. Instance-dependent Early Stopping needs changes in a quantity, not its raw level, to get a stable per-instance signal during training, and it reports that the signal collapses under heavy label noise. Both point toward per-instance rules built on trajectory differences, and toward checking whether the signal survives noisy settings. Finally, the probability-mass stopping rule of Robust Beam Search for ASR shows why principled rules matter. At very large beam sizes, a heuristic stopping rule's word error rate collapsed from 7.5% to 15.7%, while the probabilistic rule held at 7.2%.
Research Program
The first step is to log trajectories. We run the depth-refinement procedure of Refining Over Resampling to a generous maximum depth ($D = 6$) using Qwen2.5-1.5B and Ministral-8B, the models that paper reports. We add a distilled reasoning model of the kind used by CGES. For every round we store the extracted answer, its correctness, the draft, the critique, and token log-probabilities. These logs give, at no extra generation cost, the ground truth for every possible stopping rule, because any rule can be replayed offline.
The first analysis characterizes regression. We measure the per-round rates of wrong-to-right and right-to-wrong transitions, and we test whether regressions are predictable from observable signals. If they are not, the revert component is dropped and the contribution narrows to calibrated stopping alone.
The second step builds and certifies the rule. We fit $p_{\text{gain}}$ and $p_{\text{reg}}$ with small logistic and quantile regressors. ProS found that this model class, trained on 100–200 queries, was enough. We also fit a variant that adds hidden-state features, in the style of the Learn-to-Stop classifier, to see whether internal activations are worth the access cost. Thresholds are certified on a calibration split, and we report realized risk and mean rounds on held-out problems.
The third step stresses the exchangeability assumption. We calibrate on MATH500 and test on AMC, AIME24/25, OlympiadBench and GPQA-Diamond. This mirrors the GSM8K-to-GPQA transfer reported by Statistical Early Stopping for Reasoning Models.
The fourth step tests portability with a second, non-reasoning loop. We apply the same procedure to multi-pass grammatical error correction on BEA-2019 and CoNLL-2014. There, "correct" becomes "$F_{0.5}$ does not drop relative to full depth", and we compare directly against the MPD classifier.
Baselines
The naive reference is fixed-depth refinement at $D$, the default in Refining Over Resampling. It defines both the accuracy target and the cost ceiling. On MATH500 that paper reports Qwen2.5-1.5B at 58.0% accuracy, against 29.6% for the strongest verifier baseline. On AMC it reports a gain from 25.0% to 32.5%. We will reproduce these figures before comparing anything, and treat our reproduced numbers as the reference. We report no AIME baseline accuracy from the literature, because no reported result was found on Lacuna for per-benchmark AIME accuracy under depth refinement.
The strongest simple competitor is fixed depth tuned per benchmark: the best $T \in \{1,\dots,D\}$ chosen on the same calibration split. It captures the global diminishing-returns effect without any per-instance signal, so it is the fairest test of whether adaptivity matters at all.
The strongest existing adaptive method is answer-consistency stopping from Answer Convergence as a Signal for Early Stopping in Reasoning, transplanted from reasoning chunks to refinement rounds. It stops after $k$ unchanged answers, with $k$ tuned on calibration data. That paper reports over 40% token reduction on NaturalQuestions at maintained accuracy, but gives no per-benchmark figure for math.
Two further baselines complete the comparison. We port the MPD logistic stop classifier (Multi-pass Decoding for GEC) with a tuned threshold. On BEA-2019 it reports $F_{0.5}$ of 75.36 with T5-large while keeping 83% of one-pass speed. We also apply a CGES-style posterior over the sequence of round answers as if they were independent samples. On sampling, CGES reports 69.4% fewer calls with a 0.06-point accuracy change.
All baselines replay the same logged trajectories. They therefore share models, prompts, data and generation compute exactly, and every tunable threshold gets the same calibration split.
Two ablations isolate the proposal's ingredients. The first removes risk control: the same features with a threshold tuned for best mean accuracy, which tests whether the guarantee costs efficiency. The second removes the revert option, which tests whether regression modelling adds accuracy.
The proposal beats the baselines if, at matched mean rounds, it achieves equal or higher accuracy than tuned fixed depth and answer consistency, with realized $R \le \varepsilon$ on held-out data. It must also keep that guarantee under cross-benchmark transfer, in settings where the tuned baselines' accuracy loss exceeds $\varepsilon$.
Why not just…?
Why not just cap refinement at two or three rounds? This is the strongest threat on in-distribution data. If per-instance stopping times cluster tightly, the tuned fixed depth will match us. The cross-benchmark experiment settles the question. A depth tuned on MATH500 should under-refine AIME and over-refine easy problems. CGES observed the breadth analogue on GPQA-Diamond, where adaptive stopping used 3.61 calls on average and raised accuracy by 1.68%. If tuned fixed depth matches us on transferred benchmarks too, we will report adaptivity as unnecessary for this loop.
Why not just stop when the answer stops changing? This is cheap and probably captures much of the compute saving. It has two weaknesses. It offers no bound, and Answer Convergence itself cautions that stable answers can be wrong. It also cannot undo a regression that has already happened. We include it as a feature, so the risk-control procedure may simply certify a repetition rule. If the full rule adds nothing beyond certified repetition, the contribution becomes "calibrated repetition with a guarantee", and we will say so.
Why not just score every round with a reward model and keep the best? This needs an extra model call per round. Refining Over Resampling also reports verifier-based selection far below verifier-free refinement (29.6% against 58.0% on MATH500), consistent with reward-model miscalibration. We will run a reward-model-selection variant at matched compute to confirm this holds for selection across rounds, not just across samples.
Decisive Checks
Four results matter most. The first is the held-out risk-versus-rounds curve at $\varepsilon \in \{0.01, 0.02, 0.05\}$ on MATH500 and AMC, with realized risk lying under the target. The second is the same curve under MATH500-to-AIME/GPQA transfer, where the guarantee is formally void and we measure how far it degrades. The third is the per-round regression rate, together with the accuracy gained by the revert option. The fourth is the BEA-2019 replication against MPD, which shows whether the rule carries over to a non-reasoning refinement loop.
The idea is falsified if regressions turn out to be unpredictable, or if tuned fixed depth matches our accuracy at equal mean rounds on the transfer benchmarks.
References
- ProS: data series progressive k-NN similarity search and classification with probabilistic quality guarantees
- Progressive Similarity Search on Time Series Data
- CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistency
- Statistical Early Stopping for Reasoning Models
- Answer Convergence as a Signal for Early Stopping in Reasoning
- Multi-pass Decoding for Grammatical Error Correction
- Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
- When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version
- Instance-dependent Early Stopping
- Robust Beam Search for Encoder-Decoder Attention Based Speech Recognition Without Length Bias