Language models state false things fluently, and the tools that flag those statements tend to fall into two groups. Some are accurate but opaque, such as a probe that reads a number off the model's internal activations. Others are inspectable but weak: when ChatGPT is asked to judge whether a summary is hallucinated, it is right only 58.53% of the time (HaluEval). We propose a detector that sits between these groups. It decides whether a generated claim is fabricated by finding the moments in a stored memory where the same model was in a similar internal state while producing claims that were later checked. It then takes a similarity-weighted vote over whether those past claims turned out true or false. Each verdict comes with its evidence: "the model looked like this the last five times it invented a date, and four of those were wrong."
The opportunity comes from pairing two literatures. Case-based interpretability builds models whose predictions are, by construction, weighted votes over concrete stored examples. Hallucination research finds again and again that the information needed to detect a falsehood is present inside the model but inaccessible to the model's own self-report. A case-based detector over hidden states gives an auditor something to inspect. Because its knowledge sits in a memory rather than in weights, it can also absorb new labeled mistakes from a new domain instantly and drop mislabeled ones instantly, with no retraining.
Core Claim
A kernel vote over a labeled memory of the generator's own hidden states, taken at claim positions, can make three guarantees together. First, it can match a trained linear probe on in-domain hallucination detection. Second, it can adapt to a new domain from a few dozen newly verified claims better than a probe refit on the same claims. Third, its retrieved cases can help human auditors catch the detector's own errors rather than defer to them. The method only works if the similarity measure groups claims by the model's epistemic state (guessing versus recalling) rather than by topic or wording. The failure mode where prototypes match on texture instead of meaning (Birds look like cars) is therefore the central risk we measure, not a footnote.
Concretely, let $c$ be an atomic claim in a generation. Let $h_c$ be the generator's hidden state, averaged over the claim's tokens at a chosen layer. Let $\phi$ be a small learned projection. Let $\mathcal{M} = \{(h_i, y_i)\}$ be a memory of past claim states with labels $y_i \in \{0,1\}$, where 1 means unsupported. The detector outputs
$$ \hat{p}(c) = \frac{\sum_{i \in N_k(c)} K\big(\phi(h_c), \phi(h_i)\big)\, y_i}{\sum_{i \in N_k(c)} K\big(\phi(h_c), \phi(h_i)\big)}, $$
where $N_k(c)$ is the set of $k$ nearest memory entries and $K$ is a Gaussian kernel. This is the Nadaraya-Watson head of KernelICL moved from tabular rows to claim-level LM states. The retrieved $N_k(c)$ is the explanation, and it is faithful by construction because those cases are the only inputs to the vote.
Where This Sits
Three works supply the method. KernelICL replaces an opaque prediction head with a kernel estimator over context examples and loses almost nothing (82.87% versus 82.91% mean accuracy on TALENT). We reuse that estimator, but our "context" is a memory of verified claims and our domain is language-model factuality. AdaNPC trains an embedding with a nearest-neighbor loss and then adapts to domain shift by appending examples to its memory, with zero forgetting of source domains. We borrow both the training loss and the admission mechanism, but we admit only externally verified labels, not pseudo-labels. Self-labeling is exactly how agent memories accumulate hallucinated content (Adaptive Memory Admission Control). Rethinking Machine Unlearning via Key Deletion motivates exact removal of bad memory entries.
The closest existing work is ExaGPT. It runs nearest-neighbor search over BERT embeddings of text spans to decide whether text is machine-written, shows the matched spans to users, and raises human verification accuracy by 13.6%. It answers a different question, "who wrote this," with a different signal: surface embeddings from an external encoder. The surface signal works for authorship, because authorship is largely a property of style. Truth is not a property of style. Two claims can be near-identical in wording while one is recalled and one is invented. Our bet is that the generator's own state carries the difference. ExaGPT's design therefore becomes our most important ablation rather than our method.
Case-Based Decision-Theoretic Decoding also uses a scored case memory. It reranks whole translation hypotheses against stored (input, hypothesis, quality) triples. We work at the level of atomic claims in open-ended factual generation, and our cases are internal states rather than input–output text pairs.
Two works constrain the design. Birds look like cars shows that prototype models stay accurate (87.2% to 85.7%) even when their "explanations" are swapped for irrelevant car parts. Retrieved cases can look meaningful while matching on irrelevant features, so we must measure what the neighbors actually share. Stuck on Suggestions shows that experts anchor on algorithmic suggestions even when they are wrong. That finding rules out declaring victory just because explanations raise auditor accuracy on average. We must test whether cases help humans override wrong verdicts.
Calibrated Specificity Control gives us the downstream use. It decomposes drafts into atomic claims, offers coarser fallback versions of each claim, and calibrates thresholds on a support score. Its support estimator relies on LLM-based scoring. We plug $\hat{p}(c)$ in as that estimator, so the case-based detector controls what the final answer asserts without our having to invent a new decoding algorithm.
Research Program
The first component is the memory. We take an open model, for example a LLaMA-family 7B model, since DoLa and its code (voidism/DoLa) were evaluated there. The model generates long-form answers to LongFact prompts and short answers to TruthfulQA questions. Each output is split into atomic claims, and each claim is labeled supported or unsupported by evidence-based verification, as in the CSS pipeline. For each claim we record the mean hidden state over its tokens at several layers. DoLa's finding that factual content forms late in the network suggests where to look, and we check it rather than assume it.
For HaluEval, whose responses were not produced by our model, we pass each given response through the model token by token (teacher forcing) and record its states. That variant measures "does this model's state look like guessing when reading this claim." We report it separately from the self-generated setting.
The second component is the similarity measure $\phi$. We train it with a nearest-neighbor loss in the style of AdaNPC, so that neighbors share labels. We add a penalty that discourages neighbors from sharing topic: a gradient-reversed classifier that tries to predict the TruthfulQA category or HaluEval task from $\phi(h)$. We report label purity and topic overlap of neighbor sets side by side as a direct, text-domain version of the "birds look like cars" diagnosis.
The third component is memory operations. These include admitting newly verified claims from a new domain, deleting entries found to be mislabeled, and capping memory size. Deletion is exact: removing an entry provably removes its influence on the vote.
Baselines
The do-nothing reference is the generator's own token probability for the claim, averaged over its tokens and used as a confidence score. It costs nothing, and any method that cannot beat it is not worth its memory.
The strongest competitor on the detection question is a linear probe trained on the same hidden states, the same layers, and the same labeled claims. It sees exactly the information our method sees, and the truth-tracking-heads line of work (The Truth Stays in the Family) shows that truth is readable from internal components. No reported result was found on Lacuna for a hidden-state probe on HaluEval or LongFact claims. We will therefore train it ourselves, giving it the same tuning budget as our $\phi$: the same number of hyperparameter trials, the same validation split, and the same early stopping. Beating it means matching its in-domain AUROC within one point while exceeding it by a clear margin, which we pre-register as three points of AUROC, in the few-shot domain-transfer setting below.
The self-report baseline is the model prompted to judge each claim, in the HaluEval protocol. For context, ChatGPT reports 62.59% (QA), 72.40% (dialogue), and 58.53% (summarization) detection accuracy on HaluEval.
The key ablation is a surface-embedding kNN in the ExaGPT style: the same memory, labels, and kernel, but with claims embedded by a sentence encoder instead of the generator's states. This isolates whether the generator's own state is the ingredient that matters. A second ablation removes the topic penalty from $\phi$.
For the generation use, the reference is LongFact without intervention, which reports an Overcommitment-Aware Utility (OAU) of 0.846. Our main comparison is Calibrated CSS at OAU 0.913 with support precision 0.9865 (Answer Only as Precisely as Justified). We rerun CSS on our model's drafts with its LLM support estimator and then with ours, using the same calibration split. On TruthfulQA we also report DoLa, which the DoLa paper credits with a 17-point absolute gain on LLaMA-7B, as the strongest no-memory decoding method.
Why not just…?
Why not just train a linear probe on the hidden states? This is the real threat, and we expect it to tie in-domain. KernelICL's near-identical accuracy to its MLP head suggests kernel heads cost little but gain little in raw accuracy. Our claim rests on two things a probe lacks: cases a human can inspect, and adaptation by memory admission. Both get decisive tests. The transfer experiment refits the probe on exactly the same new-domain labels we admit to memory. If the refit probe matches our method, the adaptation argument fails and the contribution shrinks to explanations alone. In that case, explanations alone must then survive the human audit described below.
Why not just retrieve evidence and check claims against it, as in retrieval-augmented fact-checking? When a reliable evidence corpus exists, that approach is likely better, and we do not claim otherwise. Our detector is meant for settings where no such corpus is at hand at generation time. It is also complementary, since it can choose which claims are worth an expensive evidence lookup. Prepending retrieved text does not prevent the model from overriding it with its prior or from swapping entities (Milecki; Ni et al.). A detector that reads the model's state can therefore still catch errors that slip through retrieval. We test this by running the detector on retrieval-augmented outputs as well.
Why not just do nearest-neighbor search over sentence embeddings, as ExaGPT does? That design is cheaper and model-agnostic. Our hypothesis is that it groups claims by topic and phrasing, which the "birds look like cars" result predicts, and so it cannot separate a recalled fact from a confident guess with the same wording. The surface-embedding ablation tests this directly. If it matches the hidden-state version, the generator's internals are unnecessary, and we would recommend the simpler system.
Why not just ask the model whether it hallucinated? HaluEval reports detection near chance for summarization and below 50% for several open models. It also finds that showing a correct and a hallucinated answer together makes judgment worse. We include the self-report baseline for completeness rather than because we expect it to compete.
Proposed Experiments
The in-domain check trains and evaluates on held-out claims from the same distribution: LongFact claims, TruthfulQA answers, and HaluEval's QA, dialogue, and summarization tasks. We report AUROC and accuracy at a fixed false-positive rate, borrowing ExaGPT's emphasis on the strict 1% FPR operating point, where opaque detectors were weakest.
The transfer check is the decisive one for the adaptation claim. We hold out TruthfulQA categories, for example health and law, and HaluEval tasks, for example training on QA and testing on summarization. We admit $n \in \{0, 16, 64, 256\}$ verified new-domain claims to memory and refit the probe on the same $n$, then plot both curves.
The neighbor-semantics check reports, for each method, how often the top-$k$ neighbors share the query's label versus its topic. A method whose neighbors are mostly same-topic, mixed-label cases is producing car-part explanations, however accurate it looks.
The deletion check corrupts 10% of memory labels, then deletes the corrupted entries. It verifies that detection returns exactly to its uncorrupted performance and measures how much a refit probe degrades under the same corruption.
The human audit gives annotators claims with one of three aids: no aid, the detector's score alone, or the score plus its top three cases. A third of the items are deliberately chosen where the detector is wrong. Following Stuck on Suggestions, the primary outcome is not overall accuracy. It is the rate at which annotators correctly override wrong verdicts. Cases that raise agreement with the detector without raising correct overrides would be evidence of anchoring, and we would report them as such.
The downstream check swaps our score into Calibrated CSS on LongFact and reports OAU, support precision, and specificity retention against the 0.913 / 0.9865 / 0.938 reference. Matching CSS's OAU with a cheaper support estimator that also yields inspectable cases counts as success; exceeding it is a bonus.
The project fails cleanly if three things all hold: the surface-embedding ablation ties the hidden-state detector, the refit probe ties memory admission, and cases do not raise correct overrides. Each of those outcomes would also be a useful negative result for anyone tempted to bolt nearest-neighbor explanations onto hallucination detectors.
References
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- Birds look like cars: adversarial analysis of intrinsically interpretable deep learning
- Inspectable Tabular Foundation Models via In-Context Kernel Learning
- AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation
- Adaptive Memory Admission Control For LLM Agents
- Rethinking Machine Unlearning: Models Designed to Forget via Key Deletion
- ExaGPT: Example-Based Machine-Generated Text Detection for Human Interpretability
- Case-Based Decision-Theoretic Decoding with Quality Memories
- Stuck on Suggestions: Automation Bias, the Anchoring Effect, and the Factors That Shape Them in Computational Pathology
- Answer Only as Precisely as Justified: Calibrated Claim-Level Specificity Control for Agentic Systems
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
- The Truth Stays in the Family: Enhancing Contextual Truthfulness via Inherited Heads in Model Lineages
- MEDIMP: 3D Medical Images and clinical Prompts for renal transplant representation learning
- CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools