Large language models often "know" when a statement is false even while saying it. A simple linear classifier (a probe) trained on the model's internal activations can read that knowledge out and flag likely hallucinations. These probes are fit on small, hand-picked labeled sets such as TruthfulQA questions, balanced true/false pairs, or synthetic contradictions. They are then used on real traffic whose topics, style, and hallucination rate look nothing like the training set. This is the classic convenience-sample problem from epidemiology, and it has a known failure: the classifier learns whatever happens to separate the labels in the sample. Often that is topic or source rather than truth. We propose to find the directions in activation space that encode how the calibration data was sampled, using principal component analysis (PCA, which finds the axes along which a set of vectors varies most) on unlabeled deployment activations where the sampling confound is broken. We remove those directions before fitting the probe, and we correct the probe's decision threshold for the deployment base rate.
In two sentences, the claim is this. Some of the hallucination-detection failures blamed on the model come from the probe's own biased calibration sample, and those failures can be repaired without new labels. Other failures come from the model genuinely believing a falsehood, and those cannot be repaired this way. We will build the tool that separates the two cases and measure how much of the observed transfer failure each one accounts for.
Core Claim
Three findings motivate the proposal. First, the truth signal is tiny. HalluTracer shows that it occupies under 1% of residual-stream variance, and it averages per-layer probe scores across depth to suppress noise. Second, when falsehoods are backed by strong spurious associations learned in pretraining, entropy, perplexity and hidden-state probes all fall toward chance AUROC, according to When Bias Pretends to Be Truth. Third, a probe fit on the remaining 99% of variance has every opportunity to latch onto a nuisance axis that correlates with the label in the calibration set.
Wang et al. study bias in the model's training data. They do not ask whether bias in the probe's calibration sample produces a similar collapse, and neither HalluTracer nor A Single Direction of Truth examines how topic skew in probe training affects transfer. Our hypothesis is that a meaningful share of cross-benchmark probe failure is calibration confounding. We also hypothesize that this share is removable by projection, and that the pretraining-shortcut share is not. We treat the second hypothesis as a falsifiable prediction, not an inconvenience.
Method
Let $h \in \mathbb{R}^d$ be the answer-onset activation at a given layer. We have a small labeled calibration set $C$ with known metadata $g$ (source benchmark, topic category, template). We also have a pool $D$ of unlabeled activations from the deployment distribution, tagged with the same metadata. In $D$, topic is not tied to the truth label the way it is in $C$.
We estimate a nuisance subspace $U$ (a $d \times k$ orthonormal basis) as the top $k$ solutions of a generalized eigenproblem:
$$\Sigma_{\text{between}}^{D}\, u = \lambda\, \Sigma_{\text{within}}^{D}\, u .$$
Here $\Sigma_{\text{between}}^{D}$ is the covariance of metadata-group means in $D$, and $\Sigma_{\text{within}}^{D}$ is the average within-group covariance. This amounts to marginalizing covariance by the sampling variable, in the spirit of Demixed Principal Component Analysis. That method splits neural covariance into task-parameter-specific pieces; we reuse its marginalization idea for sampling variables rather than stimulus variables. Where the metadata $g$ is itself unobserved, we substitute the calibration-versus-deployment shift. In that case $\Sigma_{\text{between}}$ becomes the covariance of the two sample means plus the leading eigenvectors of $\Sigma^{D} - \Sigma^{C}$. This addresses the "selection variables are latent" problem noted in the sampling-bias literature.
The probe is then logistic regression on $(I - UU^\top)h$ at every layer, depth-averaged as in HalluTracer. The final step is a prior-shift correction. We estimate the deployment hallucination rate $\pi_D$ on $D$ with the standard expectation-maximization prior-adjustment procedure and shift the probe's logit by $\log\frac{\pi_D}{1-\pi_D} - \log\frac{\pi_C}{1-\pi_C}$. This is the neural-probe analogue of the base-rate effect in Abu-Akel et al., where the optimal diagnostic cutoff moved from 31.5 to 34.2 when an enriched sample was replaced by a prevalence-true one.
The PCA-compression papers supply the evidence that the relevant structure is low-rank and linearly removable. Loki finds that about 90% of key variance lies in 60–80 of 128 dimensions. La RoSA concentrates activation variance with calibration-set PCA. The same papers also constrain the design. Ma et al. show that PCA fit on one domain degrades out of domain, which is why $U$ is estimated on the deployment pool rather than on WikiText-style calibration text. Raunak et al. remove dominant principal components without regard to what they encode. We remove only the directions tied to sampling variables, and a key ablation tests whether that selectivity matters.
Research Program
The work has three parts.
The first part is a controlled confounding study. We build probe calibration sets in which the truth label is correlated with a nuisance variable at a chosen strength $\rho_c \in \{0.5, 0.7, 0.9, 1.0\}$. The nuisance variables are TruthfulQA category, HaluEval task (QA versus dialogue versus summarization), or a surface template. We evaluate on deconfounded held-out data. This measures how fast standard probes collapse as calibration confounding grows, and whether projection recovers them.
The second part is a two-source decomposition on the synthetic biography corpus of When Bias Pretends to Be Truth. There we vary pretraining spurious correlation $\rho$ and probe-calibration confound $\rho_c$ independently, on a small model trained from scratch or fine-tuned. We predict that projection restores AUROC along the $\rho_c$ axis but not along the $\rho$ axis. If it also restores AUROC along $\rho$, that contradicts Wang et al.'s claim that the model internally endorses the shortcut and would be a striking finding. If it restores neither, our core claim is wrong.
The third part is a natural transfer study. We train on one benchmark and test on the others, across TruthfulQA, HaluEval QA and summarization, TrueFalse, and CONTRATALES. The models are LLaMA-2-7B, LLaMA-3.1-8B and Qwen2.5-7B, the families HalluTracer used. Each test set is re-sampled at hallucination base rates of 50%, 19.5% (the ChatGPT general-query rate reported in HaluEval), and 5%.
We report AUROC, expected calibration error, and precision at fixed 80% recall. AUROC alone ignores the thresholding error we care about, so the last two are required.
Baselines
The naive reference is output-level uncertainty: mean token entropy and sequence perplexity, thresholded on the calibration set. It needs no activations and marks the floor. Wang et al. already show it collapses under spurious correlation, so we expect it to be weak in the second part and moderate elsewhere.
The strongest existing detector is HalluTracer's depth-averaged per-layer probe (HalluTracer). It is the strongest competitor because it already addresses the other main source of probe brittleness, per-layer noise, and our method is built on top of it. The most relevant reported numbers are 97.86% AUROC on the Agentic benchmark with Qwen2.5-72B, and a 1 to 14 point AUROC gain over single-layer probes on TruthfulQA. The file does not give an absolute TruthfulQA AUROC, and there is no reported result found on Lacuna for HalluTracer on HaluEval2. We will therefore re-run it ourselves on identical activations, layers, labeled examples and regularization search.
We also include the Observer-style single-direction residual probe from A Single Direction of Truth, which reports 0.84 F1 on CONTRATALES. This is a second supervised-probe reference on the one benchmark where a published number exists.
The fair classical competitor is importance reweighting. We fit a classifier that predicts calibration-versus-deployment membership from metadata or activations, then weight calibration examples by the resulting density ratio. It gets exactly the same metadata and unlabeled pool as our method. Prior-shift correction is applied to every probe baseline, so that our threshold correction is not credited as a representation gain.
Three ablations isolate the key ingredient. The first removes the top-$k$ unconditional principal components of $D$ (Raunak-style purification) instead of the sampling-variable subspace. The second estimates $U$ on $C$ instead of $D$. The third uses a random $k$-dimensional subspace.
Beating HalluTracer means a statistically reliable gain in out-of-distribution AUROC and in precision at 80% recall at confound strengths $\rho_c \ge 0.7$, with no loss beyond one point at $\rho_c = 0.5$. All results are averaged over five resampled calibration sets.
Why not just…?
Why not just reweight the calibration set to match deployment? This is the most serious threat, and it may win whenever every metadata group contains both labels. Reweighting fails when support is missing. At $\rho_c = 1.0$ some groups contain only one label, so no weight can recover the missing cell, while projection uses the unlabeled pool to identify the nuisance axis anyway. Reweighting also inflates variance when density ratios are extreme. The confounding sweep maps exactly where each method wins. If reweighting matches projection at every $\rho_c$, we will report that the representation-level step is unnecessary.
Why not just strip the top principal components, as in embedding purification? Raunak et al. show that dominant components hurt word-similarity tasks, but nothing guarantees those components are the confounded ones. The truth signal is small, yet it may still load on high-variance directions. The unconditional-PCA ablation tests this directly. If it performs as well as the sampling-variable subspace, the selectivity claim fails.
Why not just collect a representative, balanced calibration set? That is the textbook remedy, and representative sampling stays the gold standard (Naveed et al.). But labeling hallucinations across every deployment topic is expensive, and it is exactly what practitioners avoid. We will measure how many representative labeled examples a plain HalluTracer probe needs to match our method trained on the confounded set. That gives a concrete label-savings figure rather than an argument.
Why not just use entropy or refusal training? When Bias Pretends to Be Truth reports that both fail on confident, bias-driven errors. Entropy is included as the naive baseline so this is checked on our data too.
Decisive Checks
The idea stands or falls on two measurements. The first is the confounding sweep: at $\rho_c = 0.9$, does the deconfounded depth-averaged probe close most of the out-of-distribution AUROC gap between the confounded HalluTracer probe and a HalluTracer probe trained on an unconfounded calibration set? The second is the two-source decomposition on synthetic biographies: does projection recover performance lost to calibration confounding while leaving the pretraining-$\rho$ collapse intact? A positive result on both would give practitioners a label-free repair, together with a diagnostic that tells them which of their failures the repair can fix. Failure on the first check would show that probe transfer problems are not mainly a sampling artifact, which is also worth knowing.
References
- HalluTracer: Hallucination Detection via Depth-Averaging Truth Signals
- When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
- A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinations
- Demixed Principal Component Analysis
- The distribution of autistic traits across the autism spectrum: evidence for discontinuous dimensional subpopulations underlying the autism continuum
- Loki: Low-rank Keys for Efficient Sparse Attention
- La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation
- Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval
- Effective Dimensionality Reduction for Word Embeddings
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- The effect of women’s development on the relationship between the social impact of green microfinance institutions and poverty in Pakistan