When a language model answers a hard question by sampling many reasoning paths and taking a majority vote (self-consistency), the usual way to save compute is to stop once the sampled answers agree. That rule has a known blind spot. When the model is confidently and repeatedly wrong, every path gives the same bad answer, agreement looks perfect, and sampling stops at once. This proposal asks whether the hidden activations of those paths can separate genuine agreement from this kind of repeated error. The activations are internal vectors of 1024 to 8192 numbers per path, and a query has only a handful of paths. The question is whether their spread, measured with a properly regularized covariance, tells us when to trust a unanimous vote and when to spend more compute on sampling that pushes the paths apart.
The obstacle is statistical, and it is why nobody has a clean answer yet. With $k \le 64$ paths in a space of dimension $p \ge 1024$, the covariance of those vectors is singular. At least $p-k+1$ of its eigenvalues are exactly zero, so volumes, log-determinants, and Mahalanobis distances (distances that rescale each direction by how much the data varies along it) are all undefined. On top of that, raw activation space is dominated by a few very large coordinates, so plain Euclidean or cosine spread mostly measures those coordinates. The plan is to borrow two remedies from covariance estimation. The first is shrinkage: blend the noisy sample covariance with a simple well-conditioned target so it becomes invertible. The second is whitening against a reference covariance, which rescales every direction to comparable variance. Together they make rollout geometry measurable, and the resulting signal can drive how many samples each query gets.
Core Claim
A regularized log-volume of a query's rollout activations can be computed exactly and cheaply in $k$ dimensions rather than $p$. It is computed after whitening with a Ledoit–Wolf-shrunk reference covariance estimated from unlabeled calibration traces. The claim is that this quantity, together with answer agreement, predicts whether a unanimous self-consistency vote is correct. Using it to gate early stopping, and to redirect budget toward diversity-promoting sampling when the vote is unanimous but the geometry is suspicious, should give better accuracy per sample than surface-level stopping rules. The gains should be largest on exactly the cases where those rules fail by construction.
Why This Is Not Already Done
The closest work is Test-time Diverse Reasoning by Riemannian Activation Steering (SPREAD). It treats rollout activations as the edges of a parallelepiped (a slanted high-dimensional box) and maximizes that box's log-determinant volume to steer generation toward diverse paths. It handles the $k \ll p$ singularity with unit-sphere constraints and Riemannian coordinate descent. It uses volume only as a steering objective, never as a signal for how much compute to spend, and it does not use shrinkage or whitening. We reuse its insight that reasoning collapse happens along correlated activation directions, and we reuse its method as the intervention applied once our signal fires.
On the allocation side, Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning (ESC) stops when a window of $w$ samples is unanimous. Its authors name repeated confident error, which they call "consistent hallucination", as its main failure mode. Dynamic Early Exit in Reasoning Models (DEER) exits based on token confidence and needs per-model threshold tuning. Thought calibration trains linear probes on single-trace hidden states with Learn-then-Test risk control, a calibration procedure that bounds the error rate of the stopping rule. Its token savings fall from up to 60% in distribution to 20% on AIME and GPQA. All three read a scalar per trace or per vote. None looks at how a set of rollouts is spread in activation space.
The motivating evidence for looking there comes from Tracing Uncertainty in Language Model "Reasoning". It finds that incorrect traces often carry low epistemic uncertainty, meaning the model is not unsure in the way uncertainty measures detect. Scalar self-certainty is weak there, while a richer 30-feature profile reaches an AUROC of 0.807 for predicting correctness. The estimation machinery comes from vision. A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation shows that shrinkage of a pooled covariance makes Gaussian discriminant analysis usable when there are far fewer samples than dimensions. A Bayesian Nonparametric Perspective on Mahalanobis Distance for Out of Distribution Detection shows that structured covariance shrinkage beats unregularized relative Mahalanobis scores when per-class samples are scarce relative to the embedding dimension. Neither paper touches sequential generation or compute allocation.
The Estimator
Let $h_1,\dots,h_k \in \mathbb{R}^p$ be one layer's hidden states at the final-answer position of $k$ rollouts for a query. Offline, we collect $m$ such states from unlabeled rollouts on calibration prompts, with $m$ in the low thousands and still of the same order as $p$. From them we form the shrunk reference covariance
$$\hat\Sigma = (1-\alpha)S_{\text{ref}} + \alpha \tfrac{\operatorname{tr}(S_{\text{ref}})}{p} I_p ,$$
where $S_{\text{ref}}$ is their sample covariance and $\alpha$ is the Ledoit–Wolf intensity. Because the target is a scaled identity, $\hat\Sigma^{-1/2}$ follows directly from a thin singular value decomposition of the centered calibration matrix, computed once per model and layer. Each rollout is whitened as $z_i = \hat\Sigma^{-1/2}(h_i - \bar h)$, where $\bar h$ is that query's rollout mean. The query statistic is the ridge-regularized log-volume
$$V_k = \log\det\!\big(\lambda I_k + Z^\top Z\big) - k\log\lambda ,$$
with $Z = [z_1,\dots,z_k]$ and ridge $\lambda>0$. By Sylvester's determinant identity this equals $\log\det(\lambda I_p + ZZ^\top) - p\log\lambda$. It is therefore exactly the log-determinant of a regularized $p$-dimensional covariance, which is undefined without the ridge because the $p$-dimensional covariance is singular. The identity lets us compute it from a $k\times k$ Gram matrix at cost $O(k^2 p)$, which is negligible next to generating a single token.
Two ingredients carry the proposal: whitening against a shrunk reference, and a ridge inside the log-determinant. Everything else is a thin control layer on top.
Research Program
The first stage is diagnostic and needs no new generation machinery. For each model we generate $N=40$ rollouts per question, following ESC's budget. We record final-answer hidden states at several layers and label correctness. Then we isolate the subset of windows where ESC would stop, meaning $w$ consecutive unanimous answers. In that subset answer entropy is identically zero, so any entropy- or agreement-based score has AUROC 0.5 by construction. We ask whether $V_w$ separates correct from incorrect unanimous windows.
The sign of the relationship is an open empirical question, and we do not assume it. SPREAD's collapse picture predicts that confident errors are geometrically collapsed. The alternative is that correct consensus reached by different routes is the more spread-out case. The stopping rule is therefore calibrated on a small labeled split of about 200 questions, which chooses the direction and threshold. We then apply Learn-then-Test, as in thought calibration, so that the rate of accepting a wrong unanimous answer is bounded at a chosen level.
The second stage turns the signal into allocation. Sampling proceeds in windows as in ESC. When a window is unanimous and $V_w$ falls in the calibrated safe region, we stop. When a window is unanimous but geometrically suspicious, the remaining budget goes to diversity-promoting sampling (SPREAD steering, or a raised temperature as the cheap variant) rather than more samples from the same collapsed distribution. Non-unanimous windows continue as in ESC. The final answer is the majority over all samples.
One honest caveat shapes the design. Learning How Hard to Think shows that some queries are effectively impossible, with flat marginal reward curves. Extra compute on them is wasted. We therefore report separately how often flagged queries are rescued and how often they are merely re-sampled without benefit. We also test a variant that abstains or returns early on flagged queries whose diversified samples still fail to produce a competing answer.
The third stage tests robustness under distribution shift. The reference covariance and thresholds are fit on GSM8K and MATH traces and applied unchanged to AIME 2024 and GPQA Diamond. This mirrors the in-distribution versus out-of-distribution gap reported by thought calibration.
Baselines
The naive reference point is full-budget self-consistency with 40 samples per question, the fixed allocation that every adaptive method tries to beat. It sets the accuracy ceiling for majority voting and the cost ceiling.
The strongest training-free competitor is ESC, run with the same window size, the same underlying samples, and the same maximum budget. Stopping decisions are replayed on identical cached rollouts, so differences come only from the stopping rule. ESC reports a 80.1% sample reduction on GSM8K and a 33.8% reduction on MATH, with accuracy within 0.1% of full self-consistency (Escape Sky-high Cost). Beating it means strictly higher accuracy at matched average samples on MATH-500 and AIME. Equivalently, it means fewer wrong unanimous stops at the same sample savings.
The strongest supervised competitor is a thought-calibration-style linear probe on single-trace hidden states. It is given the same 200 labeled calibration questions as our threshold, plus the same unlabeled pool if it can use it, and the same Learn-then-Test risk level. That paper reports up to 60% token reduction in distribution and a 20% reduction on out-of-distribution AIME and GPQA (Thought calibration). The file has no accuracy-at-matched-samples figure for that probe on these benchmarks, so we will measure it ourselves. The test of our method is the out-of-distribution split: we should keep more of the in-distribution savings than the probe does at the same bounded error.
For allocation across queries we include the marginal-reward allocator of Learning How Hard to Think, which reports a 50% compute reduction and up to 10% higher success at fixed budget. We run it at its recommended roughly 3,000 labeled queries and again at our 200. This shows whether the unlabeled-geometry approach wins specifically in the low-label regime. We also report ZIP-RC's reported gain of up to 12% absolute over majority voting on AIME (Zero-Overhead Introspection) as an external reference. It requires training the model's own vocabulary head, so it is not a matched comparison.
Ablations isolate the key ingredient. We replace $\hat\Sigma$ with the identity, which gives raw activation volume. We replace it with a diagonal covariance. We drop shrinkage and use a pseudo-inverse of $S_{\text{ref}}$. We randomly project to $k<n$ dimensions before an unregularized determinant. We replace $V_k$ with the whitened trace, which measures total spread with no dependence on direction. If the identity or trace ablations match the full method, the covariance machinery is unnecessary, and we will say so.
Why not just…?
Why not just use answer entropy or windowed agreement, as ESC does? On the cases this proposal targets, unanimous windows, entropy is zero whether the answer is right or wrong. ESC's authors name this as their main vulnerability, so no refinement of the threshold can help there. The diagnostic stage measures directly how much of the remaining error mass sits in unanimous-but-wrong windows. If that fraction is tiny for a given model and benchmark, the proposal has little to gain there and we will report it.
Why not just measure raw pairwise cosine distance between rollout activations? This is the most serious threat, because it is cheap and needs no covariance. Our reason for doubting it is that raw activation distances are dominated by a few very large coordinates. Spread along the low-variance, correlated directions where SPREAD locates collapse would then be invisible. That is an argument, not evidence. The identity-whitening and whitened-trace ablations settle it on the same cached rollouts. If raw cosine dispersion reaches the same AUROC on unanimous windows, we drop the covariance estimator and report the simpler signal.
Why not just train a probe on each trace's hidden state? Thought calibration does this and loses much of its savings out of distribution. A single-trace probe also cannot see relations between traces, which is what separates consensus from collapse. The low-label and out-of-distribution comparisons test this. The probe may well win in distribution with ample labels, and we expect to concede that regime.
Why not just project to a random low-dimensional subspace so the ordinary covariance becomes invertible? With $k=8$ to 16 rollouts, the projection must go below 16 dimensions. That discards almost all directional structure, and near the sample boundary the eigenvalue noise is worst. The random-projection ablation, averaged over projection draws, quantifies this against the ridge log-volume, which keeps all $p$ coordinates.
Proposed Experiments
We use two open models whose hidden states are accessible and which appear in the cited work: Qwen2.5-1.5B (used by SPREAD) and a Llama-8B model (used by DRQA). If budget allows, we add a distilled DeepSeek-R1 reasoning model for long chain-of-thought traces. Benchmarks are GSM8K, MATH-500, AIME 2024, GPQA Diamond, and CommonsenseQA, all used by the allocation papers in this area. For each we cache 40 rollouts per question with hidden states at four layers. All stopping and allocation rules are then evaluated by replay on identical samples. The only new generation is the diversified sampling triggered by flags.
Metrics are accuracy as a function of average samples per question, and the rate of wrong unanimous stops. We also report realized error against the Learn-then-Test target, and the fraction of flagged queries actually rescued by diversified sampling.
The decisive check comes first and is cheap. On the unanimous-window subset of MATH-500 and GSM8K, whitened $V_w$ must reach AUROC clearly above 0.5 for correctness. The target is at least 0.65, reasoning that anything weaker cannot beat ESC after the stopping rule's error is bounded. It must also beat the raw-cosine and whitened-trace ablations by a margin that holds across bootstrap resamples. If either condition fails, the geometric-consensus hypothesis is wrong for these models and the allocation stages are not run.
If the check passes, the second decisive result is the matched-compute accuracy curve against ESC on AIME and MATH-500. The third is the out-of-distribution transfer to AIME and GPQA against the supervised probe under identical labels and error bounds.
References
- Test-time Diverse Reasoning by Riemannian Activation Steering
- Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning
- Dynamic Early Exit in Reasoning Models
- Thought calibration: Efficient and confident test-time scaling
- Tracing Uncertainty in Language Model "Reasoning"
- A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation
- A Bayesian Nonparametric Perspective on Mahalanobis Distance for Out of Distribution Detection
- Learning How Hard to Think: Input-Adaptive Allocation of LM Computation
- Zero-Overhead Introspection for Adaptive Test-Time Compute