Language models used as automatic graders ("LLM judges") can be swayed by surface cues that have nothing to do with answer quality. Labeling an answer as written by an "EXPERT", dating it 2025 instead of 1950, adding a fabricated citation, or padding it with extra length all flip verdicts, and the judge's written explanation never mentions the cue. We propose reading the judge's verdict from its hidden activations instead of its text output. The score is a single linear direction found by Fisher's linear discriminant, the classical method that picks the direction separating two classes most widely relative to how spread out each class is. The twist is what counts as "spread". Alongside ordinary variation among good answers, we also count the activation changes caused by swapping a shortcut cue on an otherwise identical answer. The resulting direction separates good from bad answers while staying, by construction, nearly flat along every known shortcut at once. The whole thing is one regularized linear solve on a few hundred labeled examples, with no fine-tuning.
The opportunity is that two literatures each hold half of the answer. Representation studies show that correctness lives in a small linear subspace of a model's activations, and that linear readouts beat output-based confidence by a wide margin. Bias studies show that judges' shortcut sensitivity is large and invisible in their explanations. A third result sharpens the stakes: suppressing one shortcut tends to push reliance onto a correlated, unmonitored one. Our central question is therefore not only whether a Fisher readout of a judge is accurate. It is whether suppressing several shortcuts jointly in closed form leaves the judge no more sensitive, and ideally less sensitive, to shortcuts we never showed it.
Core Claim
Let $h(x)$ be the judge's hidden state at the verdict position after it reads a prompt and one candidate response $x$. Let $\mu_+$ and $\mu_-$ be the mean states for acceptable and unacceptable responses, and let $S_w$ be the usual within-class scatter. For a set of counterfactual pairs $(x, \tilde x)$ that differ only by an injected cue (provenance label, date, fake citation, padding, politeness), define the cue scatter $S_c = \sum (h(x)-h(\tilde x))(h(x)-h(\tilde x))^T$. The scoring direction maximizes
$$J(w) = \frac{(w^T(\mu_+-\mu_-))^2}{w^T\,(S_w + \gamma S_c + \lambda I)\,w}, \qquad w^\star \propto (S_w + \gamma S_c + \lambda I)^{-1}(\mu_+ - \mu_-),$$
where $\gamma$ sets how strongly cue-induced movement is penalized and $\lambda$ is a ridge term. The ridge term is needed because hidden sizes of several thousand exceed the number of labeled examples, which makes $S_w$ singular. A pairwise verdict is $\mathrm{sign}(w^{\star T}(h(A)-h(B)))$. Each response is encoded separately, so the order in which responses are shown cannot bias the verdict, and the cost is one forward pass per response.
What is new is the cue scatter term. It turns "do not depend on this shortcut" into within-class variance that Fisher's criterion already knows how to suppress, for many shortcuts simultaneously. It also yields a testable prediction. If shortcut directions in the judge's activations are roughly linear and partly shared, then penalizing a few known cues should also reduce sensitivity to held-out cues. A single-cue removal should not have this effect, and may increase held-out sensitivity, in the spirit of bias substitution.
Relation to Prior Work
Three papers motivate the design. The Silent Judge injects provenance and recency cues into ELI5 and LitBench pairs. It measures verdict shift rates up to +30% for GPT-4o, with a 0% rate of the cue being acknowledged in explanations. Justice or Prejudice? (CALM) catalogs twelve biases, including authority via fabricated citations, verbosity, and sentiment. Both papers diagnose behavior and propose no representation-level fix, which is the gap we fill. The Confidence Manifold finds that correctness in generators' activations sits in a 3–8 dimensional subspace driven by a mean shift. Its linear probes reach 0.80–0.97 AUC, against 0.44–0.64 for output-based confidence. It studies models answering questions rather than judges grading answers, and it never separates quality directions from shortcut directions.
Two papers supply or constrain the method. Activation Reward Models build a reward from raw mean activation differences in attention heads chosen by a REINFORCE search, and inject those differences as steering vectors. They report 77.24% on RewardBench with Qwen2.5-VL-7B and 78.37% on PreferenceHack's length split, where GPT-4o scores 3.91%. Their direction ignores covariance and has no explicit treatment of cues. Ours is a covariance-whitened readout with a cue penalty, found in closed form. A Geometric Analysis of Small-sized Language Model Hallucinations applies Fisher analysis to external SBERT sentence embeddings, raising class separation from about 1:1 to 7.26x. That axis is prompt-specific, and the paper uses an LLM judge only as a labeler. We test whether moving to the judge's internal states removes the prompt-specificity.
Reward Bias Substitution constrains the claim. It shows that single-axis fixes rotate reliance onto correlated shortcuts: a length penalty in GRPO training of Llama-3.2-3B raised calibration error from 0.25 to 0.41 and cut TriviaQA accuracy from 56% to 42%. It also proves that static audits cannot tell a genuine fix from substitution. We take this seriously in two ways. Our evaluation includes held-out cues, and it includes a policy-induced audit in which the scorer actually selects outputs, rather than relying on benchmark accuracy alone. Classical support comes from A Closer Look at Prototype Classifier, where LDA on frozen features rivals fine-tuned heads. RDALS adds a conditioning argument for why Fisher projections are robust to estimation noise. Both results come from vision only.
Research Program
We use open-weight judges: Qwen2.5-VL-7B, for direct comparison with Activation RMs, plus Llama-3.1-8B-Instruct and Qwen2.5-32B. For each judge we extract hidden states at the verdict token across layers, using a standard judging prompt.
Quality labels come from the training splits of RewardBench-style chosen/rejected data. Cue counterfactuals are built by programmatic injection, with the response content held fixed. The cue families are provenance and recency labels as in the Silent Judge, fabricated citations and sentiment rewrites as in CALM, and non-substantive padding and formatting. We split the cue families into a training set (padding, provenance) and a held-out set (recency, fake citations, politeness). The held-out set is never used to estimate $S_c$.
The program has three analyses. First, a geometry study. We measure the angle between the quality direction and each cue direction, and test whether the cue directions span a low-rank shared subspace. The joint-suppression hypothesis needs this shared structure; if cue directions are mutually orthogonal, held-out transfer should fail, and we will report that. Second, a sample-efficiency and regularization study over $N \in \{25, 100, 400\}$ labeled pairs and a grid of $\lambda$ and $\gamma$. This includes checking whether a near-universal $\lambda$ appears across judges, as APORIA found for SBERT. Third, a transfer study. The same estimator is fit on internal states and on SBERT embeddings of the same responses, then tested on prompts and domains never seen in fitting. This tests whether internal states are what removes APORIA's prompt-specificity.
Baselines
The naive reference points are chance (50% on pairwise tasks) and the judge's own text verdict. The text verdict is scored from the verdict-token probability in a single order, and also averaged over both orders. This tells us whether reading activations adds anything over the same model's spoken answer, using the identical prompt.
The mean-difference direction, $w=\mu_+-\mu_-$, is the centroid method of the Confidence Manifold and the core of Activation RMs. It is fit on the same layer and the same labeled pairs. Beating it on held-out-cue verdict shift, at equal accuracy, isolates the value of whitening and of the cue penalty.
The published Activation RM numbers are the strongest existing few-shot white-box competitor: 77.24% on RewardBench and 78.37% on PreferenceHack length, both with Qwen2.5-VL-7B. Matching its backbone and support-set size, beating it means exceeding 77.24% on RewardBench or 78.37% on the length split. Where feasible we also rerun it on our cue suites. No reported result was found on Lacuna for Activation RMs on JudgeBench or on Silent-Judge-style cue injections.
For reference ceilings we use the trained reward model ArmoRM-Llama3-8B, which scores 89.0 on RewardBench per RewardBench. On JudgeBench we use o3-mini (high) at 80.86% accuracy, Claude-3.5-Sonnet at 64.29%, and GPT-4o at 50.86%. We do not expect to beat the trained reward model on RewardBench with a few hundred labels. We frame success there as closing a stated fraction of the gap from Activation RM, at a small fraction of the training data.
SCOPE is the uncertainty comparator. It averages verdict probabilities over both response orders, uses the entropy of that average as an uncertainty score, and accepts or abstains under conformal risk control. We report its accepted fraction at a 10% target error rate. We compare against the same conformal procedure applied to the margin $|w^{\star T}(h(A)-h(B))|$, which needs one pass per response. Beating SCOPE means accepting more judgments at the same controlled error.
Two ablations isolate the key ingredient. The first is Fisher without the cue term ($\gamma=0$). The second is single-axis removal, which projects out only the padding direction, akin to a single-axis length penalty. All linear methods share the same labeled data, layer search, and tuning budget, with hyperparameters chosen on a validation split that contains no held-out cues.
Why not just…?
Why not just prompt a stronger reasoning judge? On JudgeBench, o3-mini (high) reaches 80.86%, so reasoning judges are genuinely strong on objective correctness. But the Silent Judge shows that explanation-producing judges shift verdicts on cues with 0% acknowledgment, and nothing indicates that chain-of-thought removes cue sensitivity. We will measure verdict shift on our held-out cues for the reasoning judges we can access. If they show near-zero shift at comparable accuracy, our method's case reduces to cost and white-box auditability, and we will say so.
Why not just train a logistic probe, with cue-swapped copies as data augmentation? This is the real threat. Augmentation also tells the probe that cue changes should not change the label. We include it as a baseline, with the same counterfactual pairs and the same tuning budget. Two possible advantages remain for our method. One is that the cue penalty acts directly on the score's movement, whereas augmentation only constrains the label on discrete examples; this may matter at $N=25$–$100$. The other is that the closed form exposes $\gamma$ as an explicit accuracy-versus-invariance dial. If augmented logistic regression matches us on held-out-cue shift and accuracy, the contribution narrows to the counterfactual-scatter evaluation protocol and the geometry findings.
Why not just fine-tune the judge or a reward model? JudgeBench reports that fine-tuned judges such as PandaLM and JudgeLM fall below chance on hard objective pairs. Reward Bias Substitution shows that penalized optimization can trade one bias for overconfidence. A trained reward model such as ArmoRM is still stronger on RewardBench, and we report it as a ceiling. Our claim is not higher in-distribution accuracy. It is lower sensitivity to held-out cues per labeled example, without changing the judge's weights.
Why not just abstain on uncertain verdicts, as SCOPE does? Abstention controls error but leaves biased verdicts in place whenever the judge is confidently wrong because of a cue. It also needs both response orders and access to output probabilities. The conformal comparison above tests directly whether our margin is a better uncertainty signal.
Proposed Experiments
Accuracy is measured on RewardBench, PreferenceHack (all six splits), and JudgeBench, with AUC and ECE reported alongside accuracy. Cue robustness is measured as verdict shift rate on Silent-Judge-style ELI5 and LitBench injections and on CALM authority and sentiment attacks, separately for training cues and held-out cues.
The decisive check is leave-cue-family-out. With $S_c$ built only from padding and provenance, the full method should show lower held-out verdict shift than both the $\gamma=0$ readout and the single-axis removal, at no more than a small accuracy cost. If single-axis removal instead increases held-out shift while full Fisher does not, that is a representation-level version of bias substitution, and evidence that joint suppression avoids it.
Because static audits cannot separate a genuine fix from substitution, we add a policy-induced audit. Each scorer reranks best-of-N samples on UltraFeedback prompts. We then track off-target features of the selected outputs, including length, citation count, hedging and confidence language, and sentiment, together with TriviaQA accuracy and calibration. A short GRPO run with Llama-3.2-3B mirrors the Reward Bias Substitution setup, where a length penalty moved calibration error from 0.25 to 0.41.
Finally, the internal-versus-SBERT transfer comparison on unseen prompts tests whether internal states yield a prompt-invariant evaluation direction. Fisher on internal states should transfer across prompts where Fisher on SBERT embeddings does not.
References
- The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
- Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
- The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models
- Activation Reward Models for Few-Shot Model Alignment
- A Geometric Analysis of Small-sized Language Model Hallucinations
- Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure
- A Closer Look at Prototype Classifier for Few-shot Image Classification
- Regularized Discriminative Alignment for Deep Representations under Label Shift
- RewardBench: Evaluating Reward Models for Language Modeling
- JudgeBench: A Benchmark for Evaluating LLM-Based Judges
- SCOPE: Selective Conformal Optimized Pairwise LLM Judging