Reasoning language models often keep writing long after they already hold the right answer. A model may reach a correct intermediate conclusion, then say "Wait," reconsider, and wander. Current fixes stop generation when some confidence signal crosses a fixed threshold. That rule assumes the signal is at rest when it is read. We think this assumption is wrong. Confidence inside a reasoning trace moves in transients. It spikes just before the model revises itself, and it dips right after a change of strategy. A fixed threshold read at one instant cannot tell a spike from a settled state, much as an engineer who models a chip at thermal equilibrium cannot see the brief hot spot that damages it.
We propose to read a cheap linear probe from the model's hidden states at every reasoning-step boundary. A linear probe is a logistic regression trained on frozen internal activations. Ours estimates "would the answer be correct if the model stopped now?" Generation stops only when this estimate has settled: it stays high across a short window and has stopped changing. The threshold also depends on the stage of reasoning, loosely following the biological idea that the right dose depends on developmental stage. The two claims to test are these. First, a large share of wrong early exits happen during transients. Second, a settling rule applied to a zero-decoding probe beats both trial-answer decoding and smoothed single-threshold rules at matched token budgets.
Core Claim and Position in Prior Work
Two existing systems are the closest work, and they motivate the proposal. Dynamic Early Exit in Reasoning Models (DEER) pauses at transition cues such as "Wait," decodes a trial answer, and exits if the geometric-mean token probability exceeds a static threshold $\lambda$. It reports that in about 75% of cases a correct answer appears well before generation ends. It also reports token reductions of 19.1% to 80.1% with accuracy gains of 0.3% to 5.0% across 10 benchmarks and 11 models. SpecExit removes the decoding cost by attaching regression heads to a draft model's hidden states. These heads predict confidence, progress, and remaining length. SpecExit smooths the signals with an exponentially weighted moving average and reports up to 2.5× speedup and 66% fewer tokens on GSM8K, with accuracy within 1%.
Our proposal differs from both in three ways, and each is tested by an ablation. First, we probe the target model's own activations rather than a draft model's, and we decode no trial text. Second, the probe predicts correctness-if-stopped rather than progress or remaining length. Third, the stopping rule is a settling test with stage-dependent thresholds, not a threshold on a smoothed value. The third difference is the one that matters. If ordinary smoothing captures the benefit, the transient story is wrong, and the experiments are designed to show that.
Three other papers supply the method. LLMs Know More Than They Show found that linear probes on middle-to-late MLP-layer activations detect errors with an AUC (area under the ROC curve) of 0.85 to 0.95 in Mistral-7B and Llama-3-8B, beating logit-based uncertainty. This is our evidence that correctness is linearly readable before it shows in the output. Rethinking Calibration for Early-Exit Neural Networks showed that exit rules based on the current confidence alone waste compute on inputs where later computation will also fail. Its tiny meta-classifier uses the history of earlier exits and adds 0.001% overhead. We borrow the idea of conditioning on history, but we move it from layer depth in image classifiers to step time in generated reasoning. How is BERT surprised? shows that different layers carry different signals, with detection peaking at different depths across architectures. We therefore select the probe layer by validation rather than fixing it.
One paper constrains the design. The same hallucination paper found that truthfulness directions are specific to task and domain, which cuts against the cross-lingual transfer reported in Analyzing LLMs' Knowledge Boundary Cognition Across Languages. A probe trained on competition math may therefore fail on graduate science questions. We treat this as a central risk and test it directly instead of assuming it away.
On the biological direction, we are careful. The plant-growth models of A dynamical model of plant growth with full retroaction and Oscillations in Functional Structural Plant Growth Models show that feedback between supply and demand can create oscillations with no external clock. The oscillations paper predicts fruit set versus abortion at 89% accuracy on cucumber. We take one design lesson from this work: the decision rule should depend on developmental stage. We do not claim that reasoning compute obeys a supply-to-demand equation. The oscillation result mainly warns us that threshold-driven feedback produces cycles, which is exactly the spike-and-revise pattern we expect to find.
Research Program
We use DeepSeek-R1-Distill-Llama-8B and Qwen3-4B, the models SpecExit evaluated. We sample reasoning traces on the MATH training split (Measuring Mathematical Problem Solving With the MATH Dataset).
To build labels offline, we do the following at every step boundary $k$ (each blank-line split). We record hidden states at candidate layers. We then force the end-of-thinking marker, decode the answer, and label the prefix as correct or not. This is DEER's trial answer, used only to create training labels.
A logistic probe per layer is trained to output $p_k$, the probability that stopping at step $k$ yields a correct answer. The layer is chosen on a development split. As a control, the probe is compared against a probe on shuffled labels and against a logit-only predictor. This checks, as How is BERT surprised? and the probing literature caution, that the probe is not simply reading output uncertainty.
The first contribution is measurement. On held-out traces we locate transients: windows around transition cues where $p_k$ rises and then falls. We then count what fraction of the wrong exits made by a single-threshold rule fall inside such windows. If that fraction is small, the settling rule has nothing to fix, and we will report that.
The second contribution is the stopping rule. We exit at step $k$ if
$$ \min_{k-w \le j \le k} p_j \ge \lambda_{s(k)} \quad\text{and}\quad |p_k - p_{k-1}| \le \epsilon, $$
where $w$ is a window length in steps, $\epsilon$ bounds the most recent change, and $\lambda_{s(k)}$ is a threshold for stage $s(k)$. Stage is defined by the number of steps since the last transition cue, bucketed into a few bins. All of $w$, $\epsilon$, and the stage thresholds are tuned on the development split to a target token budget. Evaluating the probe costs one dot product per step, so no text is decoded at inference.
Baselines
The naive reference points are two. The first is full generation with no early exit, which sets the accuracy ceiling and the token cost to beat. The second is truncation at a fixed token budget, which tests whether any learned signal helps at all.
The strongest existing competitors are DEER and SpecExit. We run DEER with its trial-answer decoding and static $\lambda$. We run SpecExit's three regression heads with its moving-average smoothing, retrained on exactly our MATH-train traces. Every method gets the same models, the same labeled traces, and the same development-set tuning budget. For DEER, the reported numbers are the aggregate ranges given above (19.1% to 80.1% fewer tokens, 0.3% to 5.0% accuracy gain). For SpecExit, they are 2.5× speedup and up to 66% fewer tokens on GSM8K, with accuracy within 1%. Neither paper reports per-benchmark accuracy for AIME24 or HumanEval that we could find; for those, no reported result found on Lacuna. We will therefore compare against our own reruns.
Beating these baselines means dominating their accuracy-versus-tokens curve on MATH-500 and GPQA Diamond. Concretely, the settling rule must reach equal accuracy with fewer tokens, or higher accuracy at matched tokens, with the gap outside run-to-run variance over three seeds. It must also have lower wall-clock latency than DEER.
Three ablations isolate our ingredients. The first uses our probe with a single-step threshold, which separates the value of the probe from the value of the rule. The second uses our probe with moving-average smoothing, which is the decisive comparison for the settling claim. The third removes stage conditioning by using one global $\lambda$.
Why not just…?
Why not just use output confidence, as DEER does? DEER's signal requires decoding a trial answer at every check, and that cost grows with the number of checks. The hallucination-probing paper found that internal probes beat logit uncertainty for error detection. We will test whether this holds for reasoning prefixes by comparing, at matched token budgets, our probe against DEER's geometric-mean confidence and against a logit-entropy rule. If the logit signals match the probe, the probe adds nothing beyond a latency saving, and we will say so.
Why not just smooth the signal, as SpecExit does? This is the most serious threat. A moving average also suppresses spikes, and it may capture most of the benefit. The difference is in how each handles a spike. Smoothing lags behind: it can still cross the threshold on a long enough spike, and it delays exit after the signal truly settles. The settling rule requires both a sustained high value and a small recent change. The probe-plus-smoothing ablation settles the question directly. If it matches the settling rule within noise, we drop the transient claim and report a latent-probe version of SpecExit that needs no draft model.
Why not just set a token budget per difficulty level? MATH labels five difficulty levels, so a per-level budget is a cheap competitor. We will give the fixed-truncation baseline access to the gold difficulty level, which is information our method does not get. If a per-level budget matches us, the signal adds nothing beyond difficulty.
Why not model compute as a literal supply-and-demand balance? We have no evidence that reasoning compute has meaningful "supply" and "demand" quantities to measure. Building that model would add parameters without a test that could fail. We keep only the stage-conditioned threshold, which the third ablation tests on its own terms.
Decisive Checks
The first check is whether transients cause errors. Among wrong exits under the single-threshold probe rule, at least a substantial minority should fall inside the rise-then-fall windows. The share of wrong exits should also drop under the settling rule relative to smoothing.
The second check is transfer. We train the probe on MATH and test it without retraining on GPQA Diamond (GPQA) and on AIME24. This directly probes the task-specificity finding of the hallucination-probing paper. Failure to transfer would limit the method to one domain per probe, and we would report per-domain probes as the honest fallback.
The third check is the full comparison: accuracy-versus-tokens curves and wall-clock latency on MATH-500, GSM8K, and GPQA Diamond against every baseline and ablation above.
References
- Dynamic Early Exit in Reasoning Models
- SpecExit: Accelerating Large Reasoning Model via Speculative Exit
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
- Rethinking Calibration for Early-Exit Neural Networks
- How is BERT surprised? Layerwise detection of linguistic anomalies
- Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations
- A dynamical model of plant growth with full retroaction between organogenesis and photosynthesis
- Oscillations in Functional Structural Plant Growth Models
- Measuring Mathematical Problem Solving With the MATH Dataset
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark