Premature Compression: A Geometric Signal of Unreliable LLM Judges

Large language models are now routinely used as judges that decide which of two answers is better. These judges fail in two ways their written explanations hide. On pairs that differ only in subtle correctness, they guess: GPT-4o scores 50.86% on JudgeBench, roughly a coin flip. And they follow superficial cues such as a date or an author label without ever mentioning those cues in their rationale. We propose to detect both failures from the shape of the judge's internal computation, without labels and in a single forward pass. The measurement is intrinsic dimension: roughly, how many independent directions a cloud of hidden states actually spreads along, as opposed to the thousands of coordinates it is stored in.

The hypothesis is simple. A judge that has actually worked through the content first expands its representation of the verdict in middle layers and then compresses it late. A judge that decides from a shortcut, or guesses, commits early and compresses before it ever expands. If this "premature compression" shows up reliably in the layer-by-layer profile of each verdict, it gives an unsupervised flag for unreliable verdicts. That flag should transfer across domains and to shortcut types nobody labeled. It would also be a nonconformity score for conformal abstention that needs one pass instead of two.

Core Claim

The claim concerns the trajectory of local dimension across layers at the verdict token, not a single dimension number. Two findings motivate it. Geometry of Decision Making in Language Models measured dimension across layers in 28 open models on multiple-choice questions. Dimension rises to a peak in the middle layers and then collapses to 5–37 dimensions. Accuracy read out from intermediate layers jumps right after the peak, and few-shot prompting moves the peak earlier. Effective Reasoning Chains Reduce Intrinsic Dimensionality found that effective reasoning lowers the task's intrinsic dimension. Their dimension is a count of trainable parameters rather than a property of the activation cloud, so it motivates our hypothesis but does not supply a method.

Neither paper studies a model acting as a judge. Neither asks what the trajectory looks like when the decision is driven by a cue rather than by content. We predict that cue-driven and guessed verdicts show an earlier and lower peak followed by early compression. Content-grounded correct verdicts should show a later, higher peak and tight terminal compression.

This framing also resolves an apparent contradiction in the file. Lower terminal dimension tracks higher accuracy in Joshi et al. and in Local Intrinsic Dimension of Representations Predicts Alignment and Generalization. Yet A Comparative Study of Learning Paradigms in Large Language Models via Intrinsic Dimension finds that fine-tuned models, which have lower dimension, beat in-context learning. On JudgeBench, fine-tuned judges such as JudgeLM and PandaLM fall below chance. A single dimension number cannot separate "compressed because the problem was solved" from "compressed because a cheap feature was latched onto." Where along the depth the compression happens can.

Research Program

We measure local intrinsic dimension (LID) of a single point: how many directions its immediate neighborhood spans, estimated from its distances to nearby points. We use the maximum-likelihood estimator of Levina and Bickel. For hidden state $x$ at layer $\ell$ and a reference pool of hidden states from other judgments at the same layer, let $T_j(x)$ be the distance to the $j$-th nearest pool member. The estimate is

$$\hat m_\ell(x) = \Big(\tfrac{1}{k-1}\sum_{j=1}^{k-1}\log\tfrac{T_k(x)}{T_j(x)}\Big)^{-1},$$

where $k$ is the neighborhood size. Each verdict thus gets a profile $\hat m_1(x),\dots,\hat m_L(x)$ over the judge's $L$ layers. We summarize the profile by three numbers: peak layer, peak height, and terminal value. The score is the projection of these three numbers onto one direction, fitted on a small calibration set. We also keep the raw profile for analysis.

We reuse the per-sample LID idea from Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality. That work showed adversarial images get higher LID than clean ones at identical local density (4.36 vs 1.53 at equal kernel density 0.92) and built a detector from layer-wise LID. We differ in three ways. Our anomaly is a decision process rather than an input perturbation. The signal we expect is early compression, not inflation. And the detector should need no examples of the failure it detects.

Scikit-dimension reports that dimension estimators disagree and that consensus averaging is more stable. We therefore repeat the profile computation with the two-nearest-neighbor ratio estimator TWO-NN (Facco et al.) and with the generalized-ratio estimator GRIDE. Results that hold under only one estimator are treated as unreliable.

We run open-weight judges where all layers are visible: Llama-3.1 at 8B and 70B and one other family. Each pairwise prompt gets one forward pass, and we record the hidden state at the verdict token. Reference pools come from a disjoint split of a few thousand judgments.

Failure labels come from three sources, all cheap. The first is correctness: JudgeBench's 350 style-neutralized pairs across Knowledge, Reasoning, Math and Coding, plus RewardBench's Chat Hard and Reasoning categories. The second is position-driven verdicts, defined as pairs whose verdict changes when the two answers are swapped. The third is cue-driven verdicts. We reproduce the recency and provenance injections of The Silent Judge on ELI5 and LitBench and label a verdict cue-driven when inserting the cue flips it. That paper reports up to +30% verdict shift for GPT-4o with 0% cue acknowledgment, so text auditing cannot supply these labels. Labels are used only to evaluate, and to fit the three-number projection on a small calibration split. They are never used to train the score on the test distribution.

Finally, the score serves as the nonconformity score in conformal risk control, following SCOPE. The judge accepts a verdict only when the score clears a threshold set to bound the error rate among accepted verdicts.

Baselines

The naive reference point is to accept every verdict, or equivalently to rank by the judge's softmax probability on its verdict token. The Confidence Manifold found output-based uncertainty reaching only 0.44–0.64 AUC on generator correctness. For judges, SCOPE reports softmax and verbalized confidence losing to its entropy proxy on JudgeBench. Our score must clearly beat this floor or the project stops.

The strongest existing method for selective judging is SCOPE's Bidirectional Preference Entropy (BPE). It queries the judge on both orderings, turns the two answers into an entropy, and applies conformal risk control. At a target of α = 0.10 it holds the false discovery rate at 9.7%–9.9% on RewardBench while accepting up to 2.4× more evaluations than naive selective baselines. We run BPE on the same judges, data splits and calibration sets. We count its doubled compute explicitly. Beating it means matching its realized error rate at α = 0.10 with equal or higher acceptance using one pass, or higher acceptance at equal compute.

The strongest representation-based competitor is a supervised linear probe on the same verdict-token hidden states, trained to predict judge error. This is the method behind the 0.80–0.97 AUC in The Confidence Manifold, there measured on models answering questions rather than judging. We found no reported result on Lacuna for probes on LLM-judge hidden states. The probe gets the best layer and the same total labeled budget as our calibration set plus a training set, and we report it both in-domain and transferred.

Two ablations isolate the key ingredient. One replaces the full profile with terminal LID alone, testing whether trajectory shape matters beyond the final compression Joshi et al. already linked to accuracy. The other replaces LID with mean k-nearest-neighbor distance, a density proxy, testing whether dimension adds anything over "this point is far from the others." This repeats the Ma et al. comparison in which LID reached 98.94% AUC against 91.35% for kernel density on CIFAR-10.

Why not just…?

Why not just train a linear probe for judge error? This is the real threat, and in-domain we expect a probe to win, given how cleanly correctness separates linearly in The Confidence Manifold. Our bet is about transfer. The same paper needed a 5-dimensional projection to gain 10–14% AUC across domains, which suggests raw probes lose accuracy when the domain changes. A probe trained on one failure type also has no reason to catch another. The decisive test trains the probe on JudgeBench Knowledge plus recency cues and evaluates on Math, Coding, provenance cues and position flips. If the probe still beats the profile score there, we will report that LID is a weaker substitute. The remaining contribution would then be the descriptive finding about judge trajectories, and we will consider a probe-plus-profile combination only if it adds measurable transfer.

Why not just use SCOPE's two-pass entropy? It works and has guarantees. But it only sees disagreement between orderings, so a judge that is consistently swayed by a recency cue, or consistently guessing, in both orders will look confident to it. JudgeBench's 50.86% for GPT-4o already includes position swapping. The cue-injection split directly measures how many cue-driven flips BPE accepts compared with our score.

Why not just use a reasoning model as the judge? o3-mini reaches 80.86% on JudgeBench, and for raw accuracy this is the right answer. It does not remove the need to know which verdicts to trust, and 19% errors remain. Its internals are also closed. We treat this as a ceiling on accuracy, not a competitor for abstention, and we note that our method applies only to open-weight judges.

Why not just strip or randomize the cues? For known cues such as dates and author labels, counterfactual removal is cheaper and better, and we would recommend it. The geometric score matters only for cues no one thought to remove. Accordingly, every cue result is reported only on cue types held out from calibration.

Proposed Experiments

The first experiment is descriptive. We plot average LID profiles for correct, incorrect, position-driven and cue-driven verdicts on each judge, under all three estimators, with bootstrap intervals. The small JudgeBench subsets, Coding at 42 pairs for example, make intervals essential. The core claim is falsified if cue-driven and guessed verdicts do not peak earlier or lower than correct ones under at least two estimators.

The second experiment measures detection AUC for judge error and for cue-driven flips under the held-out-domain and held-out-cue protocol above. It compares the profile score against softmax confidence, BPE, the transferred probe, terminal LID alone and kNN distance.

The third experiment runs conformal risk control at α = 0.10 on RewardBench and JudgeBench and reports realized error rate and acceptance against SCOPE's 9.7%–9.9% at its reported acceptance. We also report sensitivity to the neighborhood size $k$ and the reference-pool size, since nearest-neighbor estimators are known to be noisy on small samples.

The project succeeds if the single-pass profile score matches BPE's error control with higher acceptance per unit compute, and beats the transferred probe on unseen domains and unseen cues. Showing that the trajectory, not terminal dimension or distance alone, carries that signal makes the result about geometry rather than about yet another detector.