Vetoing Bad Self-Corrections in Step-by-Step Reasoning

When a language model reasons step by step and then revises its own work, every revision is a bet. Some revisions fix a wrong step. Others overwrite a correct step with a wrong one, and every later step inherits that error. High-speed error-correcting decoders in optical communications face the same problem. Their cheap component decoders sometimes "miscorrect," flipping correct bits. The fix that made those decoders practical, soft-aided hard-decision decoding, works like this: the decoder passes only cheap binary decisions around, but it keeps the original measurement confidence for each bit in memory and uses it to veto proposed flips that the original evidence strongly contradicts. We propose the same thing for language-model self-refinement. The reasoning trace stays ordinary readable text. For each step, the model's own confidence from the first pass is stored as a single number. Any later edit to that step must overcome that stored confidence, with a weight that depends on which round of revision the edit comes from.

This connects three lines of work. It reuses the combining rule from soft-aided decoding. It targets the fragility of sequential reasoning, where one bad step corrupts everything after it. It also replaces uniform compute allocation, because extra resampling is spent only on steps whose stored confidence is low, and the revision loop stops once no step is both weak and contested. In two sentences: self-refinement methods fail partly because they accept harmful edits, and a stored first-pass confidence used as a round-dependent veto should cut those harmful edits at almost no cost. The same confidence tells us where to spend refinement compute, so easy steps and easy questions get none.

Core Claim

The claim has two parts, and the first can be tested before the second. First, in step-level self-refinement methods such as SSR: Socratic Self-Refine, a substantial fraction of accepted edits replace a correct step with an incorrect one. SSR breaks a reasoning trace into sub-questions, re-solves each one $M$ times, and replaces the weakest step with the majority answer. We call an accepted edit that breaks a correct step a miscorrection, borrowing the coding-theory term. Second, gating each edit on a combination of the refiner's evidence and the step's stored first-pass reliability should reduce miscorrections more than it reduces genuine corrections. The best combining weight should change across revision rounds, which is exactly what separates the fixed-weight decoder iBDD-SR (Sheikh et al.; Graell i Amat et al.) from the iteration-dependent Bayesian rule iBDD-CR (Refined Reliability Combining). In the decoding setting, iBDD-CR adds 0.36 dB of coding gain on AWGN. It even beats an idealized hard decoder that never miscorrects, because the soft value resolves cases that binary decisions cannot.

Mechanism

Let a first-pass chain of thought have steps $S_1,\dots,S_K$. When step $S_k$ is generated, we store one scalar $\ell_k$. It is a calibrated log-odds that the step is correct, computed from that step's token log-probabilities, and optionally from a small linear probe on the hidden state when weights are available. This scalar plays the role of the channel log-likelihood ratio (the decoder's stored measure of how strongly the raw signal favored a bit). It is computed once and never recomputed. In refinement round $r$, a refiner (SSR-style resampling of the step's sub-question) may propose a replacement $S_k'$. The refiner's evidence for the replacement is $\lambda_k^{(r)}$, the log-odds of the agreement fraction among its $M$ samples. The edit is accepted only if

$$\lambda_k^{(r)} - \alpha_r\,\ell_k > \tau_r ,$$

where $\alpha_r \ge 0$ sets how much the stored first-pass confidence counts against an edit in round $r$, and $\tau_r$ is an acceptance threshold. We fit $(\alpha_r,\tau_r)$ on a held-out calibration set to maximize net corrections, meaning corrections minus miscorrections, per round. This empirical curve stands in for density evolution, the decoder-design tool that tracks how error rates fall across iterations. If later rounds work on steps that are already mostly clean, a refiner disagreement there is more likely spurious, and we expect $\alpha_r$ to grow with $r$.

Compute allocation comes from the same quantity. The resampling budget for step $k$ decreases with $\ell_k$, so high-confidence steps are never re-solved. The loop halts when no step has low combined reliability. Easy questions therefore exit after the first pass, and hard questions spend budget only at their weak links.

This design is deliberately local, which is why continuous signals are used only at step boundaries. Soft Reasoning optimizes the embedding of the first generated token and reports that extending continuous optimization across many tokens overfits and degrades reasoning. SoftCoT injects continuous "soft thoughts" from an assistant model as a prefix. Both let continuous signals shape generation. Our proposal follows the decoder lesson instead: soft values never circulate. They only validate discrete decisions, which keeps the trace inspectable and avoids the instability Soft Reasoning reports.

Research Program

The work has three stages. The diagnostic stage measures how often miscorrections happen. We run SSR-style refinement and plain "re-solve the lowest-confidence step" refinement on GSM8K and MuSiQue, and label every accepted edit as a correction, a miscorrection, or neutral. On GSM8K, steps can be checked automatically because reference solutions carry calculator annotations (Training Verifiers to Solve Math Word Problems), so each arithmetic step's result can be verified against them. On MuSiQue, intermediate sub-answers come from its multi-hop chains. If miscorrections are rare, under roughly 10% of accepted edits, the veto has little to work with, and we would report that as a negative result rather than proceed.

The second stage builds the combining rule. It fits $\ell_k$ calibration and the per-round $(\alpha_r,\tau_r)$, and compares a fixed $\alpha$ against the round-dependent schedule. It also tests coarse storage: whether quantizing $\ell_k$ to 3 or 4 bits loses accuracy. This mirrors the under-0.07 dB loss reported for iBDD-CR, and it matters for long-context deployment, where per-step state should be tiny.

The third stage adds budget allocation and compares accuracy against generated tokens, end to end. We run open models that expose log-probabilities (LLaMA-3.1-8B-Instruct and Qwen2.5-3B/7B). A probe-free variant uses only log-probabilities, so it applies to APIs that return them.

Baselines

The naive reference point is single-pass zero-shot chain of thought with no refinement. SoftCoT reports 79.61% for this on GSM8K with LLaMA-3.1-8B. MITS reports 47.34% on StrategyQA with Qwen2.5-3B (MITS). We rerun it on our exact models and prompts.

The strongest competitor on this specific question is SSR. It is the closest existing step-level refinement method, and it always accepts the majority replacement for the weakest step. SSR-Plan with GPT-5-mini reports 62.00% on AIME 2025 against 37.00% for chain of thought (SSR). We will not claim to beat that number, since it uses a different model. Instead we rerun SSR on our open models with the same $M$, the same maximum number of rounds, and the same prompts. Our method beats it if it reaches higher accuracy at equal or fewer generated tokens, or equal accuracy with at least 25% fewer tokens.

Self-consistency, which samples many chains and takes a majority vote over final answers, is the brute-force competitor. It is run at token budgets matched to each refinement method. Its early-stopping variant (Escape Sky-high Cost) is the fair version of adaptive halting.

For StrategyQA, MITS is the strongest tree-search competitor. It reports 68.45% accuracy with Qwen2.5-3B at 64 seconds per problem. We compare against it on accuracy and wall-clock time, using the same model.

Query-level allocation in the style of Learning How Hard to Think serves as the allocation baseline. That paper reports a 50% compute reduction while matching best-of-$k$ accuracy. We implement it by spending the same total budget, distributed by a prompt-level difficulty probe.

Two ablations isolate the key ingredient. The first, trigger-only, uses $\ell_k$ to choose which steps to refine but accepts every majority edit. The second, fixed-$\alpha$, uses the veto with one weight for all rounds. The gap between trigger-only and the full method measures the value of the veto itself. The gap between fixed-$\alpha$ and the full method measures the value of round-dependent combining.

Why not just…?

Why not just use low confidence to decide which steps to resample, and skip the veto? This is the most direct competitor. It is essentially what MITS's entropy-driven branching and SSR's weakest-link selection already do. Triggering decides where to look but does nothing about bad edits once they are proposed, and our diagnostic stage measures how often those bad edits are accepted. The trigger-only ablation answers the question directly. If it matches the full method, the veto is useless and the paper becomes a negative result about miscorrection.

Why not just require a stricter majority, say 80% agreement, before accepting an edit? This is a serious threat. A fixed strict threshold is iBDD-SR with $\alpha=0$ and a high $\tau$. It may capture much of the benefit. The difference is that a strict threshold ignores how confident the original step was. It blocks edits to weak steps as readily as to strong ones, and so it should cost genuine corrections. We include a tuned constant-threshold baseline, with $\tau$ searched on the same calibration set. Only if the stored $\ell_k$ adds net corrections beyond it do we claim the coding-theory mechanism matters.

Why not just use self-consistency with more samples? SSR argues that parallel sampling plateaus because errors across samples are correlated, and Damani et al. show uniform sampling wastes compute on easy queries. Both are reasons to doubt it, not proof. The token-matched self-consistency curves settle it. If self-consistency dominates the accuracy-versus-tokens frontier, the proposal fails.

Why not just gate edits with a trained process reward model, a verifier that scores each intermediate step? Verifiers can approve invalid steps that merely look right (Cobbe et al.). They also need step-level training data. Our gate needs only one stored scalar per step from the model being refined. Where a verifier exists, its score can be added as a third term in the combining rule. We report that variant on GSM8K, so readers can see whether the stored confidence adds information beyond a verifier.

Proposed Experiments

On GSM8K, the decisive check is the miscorrection rate among accepted edits, alongside final accuracy. The full method should cut miscorrections by at least a third relative to SSR while keeping at least 90% of its corrections. It should also beat trigger-only at matched tokens.

On AIME 2024/2025, which require long derivations where a late harmful edit is costly, we report accuracy against thinking tokens, compared with SSR and with Thought calibration-style early stopping. Thought calibration reports a 20% token reduction on AIME without accuracy loss.

On MuSiQue and StrategyQA, we test whether the effect carries over from arithmetic to multi-hop factual steps, against MITS and ProbTree.

The fitted schedule $\alpha_r$ is itself a finding: we predict that it rises across rounds. A flat schedule would mean the analogy to iteration-dependent combining does not hold for language models, even if the veto still helps. Finally, the quantization test checks whether 3-bit storage of $\ell_k$ preserves accuracy within one point. That would show the per-step memory cost is negligible, as it is in the decoders the method borrows from.