A language model can write a confident sentence that is false. One way to catch this is to read the model's own internal signals while it processes the sentence. These detectors, called probes, are small classifiers trained on the model's hidden activations. They work well, but each one is trained on thousands of labeled examples from a single task. Every new deployment brings its own way of being wrong: a medical summarizer fails differently from a translation system or a trivia bot. In that setting someone can usually label five or ten outputs, not five thousand. The obvious alternative is to show an LLM judge a few labeled examples in its prompt. The literature finds that this backfires: in the SemEval-2024 hallucination task, adding few-shot examples to a GPT-4 judge made it worse than a zero-shot prompt with a clear definition (SHROOM-INDElab).
We propose to take the "few-shot" step in a different place. Labeled examples should not go into the judge's prompt. They should be used to fit a small convex classifier, a linear model with a closed-form solution, on top of a feature map over the model's internal hallucination signals. That feature map is meta-trained across many hallucination tasks, meaning it is trained specifically so that a classifier fitted on a handful of examples from an unseen task works well. The claim is concrete. With 4–16 labeled outputs from a held-out task, such a probe should approach a probe trained with full supervision on that task, and should beat both few-shot LLM judges and naive small-sample probes.
Core Claim and Positioning
Internal-state detection already shows that useful signals exist. In-Context Sharpness as Alerts finds that factual tokens come with sharp, low-entropy activation patterns over the prompt at an intermediate layer, while hallucinated tokens come with diffuse ones. ICR Probe tracks how much each layer's update to the hidden state follows attention versus the feed-forward layers. It trains a 16k-parameter classifier on these cross-layer scores and reports AUROC "often between 0.76 and 0.84" on HaluEval, SQuAD, HotpotQA and TriviaQA. One SPACE to Rule Them All trains linear probes across attention heads on contrastive datasets. These papers supply our input features and our supervised upper bound. None of them asks how few labels a new task needs; all assume a large labeled set per setting.
The method comes from few-shot learning. MetaOptNet meta-learns an embedding so that a linear SVM solved on the support set generalizes well. It backpropagates through the solver using its optimality conditions, which avoids unrolling the solver's iterations. We reuse this machinery unchanged. What is new is the input: per-token internal signals from a frozen LLM rather than image features. The "classes" are also new: whether a span is hallucinated within a given generation task. This is the specific gap in the file's Gap 2, and we found no work that fills it.
Two papers constrain the design. The SHROOM-INDElab result is evidence that putting labeled examples in the prompt is fragile for detection. Rectifying Demonstration Shortcut in In-Context Learning gives a likely reason: models ignore the input-label mapping in demonstrations and fall back on their prior associations with the label words. We treat that explanation as a hypothesis to test, not a fact. HaluEval shows that direct self-judgment is weak; ChatGPT reaches only 58.53% accuracy on summarization hallucination recognition.
Research Program
Features. We run a frozen open model (Llama-3-8B and Qwen2.5-7B, matching the ICR Probe setup) over each prompt–response pair, feeding in the response text token by token. We extract three signals for each response token:
- the per-layer ICR scores;
- the contextual entropy at the informative layer identified by Activation Decoding;
- a low-rank projection of that layer's hidden state.
Token features are pooled to the response or span level, depending on the benchmark's labels. Because we feed in the response rather than generate it, model-agnostic outputs such as SHROOM's can be scored by a "reader" model even when the generator is unavailable.
Meta-learning. A small encoder $g_\theta$ maps these features to an embedding. Each training episode samples one task (for example, HaluEval dialogue), a support set of $k$ labeled responses per class, and a query set. On the support set we fit a ridge or SVM classifier $w^*(\theta)$, the closed-form or quadratic-programming solution given embeddings $g_\theta(x_i)$ and labels $y_i$. We then update $\theta$ to minimize the query loss:
$$ \min_\theta \; \mathbb{E}_{\text{episodes}} \; \mathcal{L}\big(w^*(\theta)^\top g_\theta(x_q),\, y_q\big), $$
where gradients pass through $w^*$ via implicit differentiation, as in MetaOptNet.
A convex base learner is the right choice here for three reasons. Its solution is unique and stable with eight examples. It handles label imbalance through class weights. Its cost does not grow with model size, because the LLM is never updated.
Protocol. Evaluation is leave-one-task-out across a pool of tasks:
- HaluEval QA, dialogue and summarization;
- SHROOM definition modeling, machine translation and paraphrase;
- the open-QA sets used by ICR Probe;
- TruthfulQA generations labeled by its judge.
The held-out task never appears during meta-training or hyperparameter selection.
Secondary analysis. We test the demonstration-shortcut explanation for why few-shot LLM judges fail. We apply In-Context Calibration to the judge's "hallucinated / not" label tokens. If calibration recovers most of the gap between few-shot and zero-shot judging, that explains the SHROOM-INDElab anomaly. It also produces a stronger prompt baseline for our main comparison.
Baselines
The naive reference point is the zero-shot LLM judge given an explicit definition of hallucination. The SHROOM-INDElab ablation found this beat their few-shot variant. We run it with the same open model used for features, and with GPT-4-class models where the budget allows. Label-free scoring is a second naive point: raw contextual entropy used as an anomaly score with no training.
The prompt-side competitors are a few-shot judge given the same $k$ labeled examples, and that judge with In-Context Calibration. For the SHROOM-INDElab system the file gives only leaderboard ranks (fourth in the model-agnostic track, sixth in the model-aware track). No reported accuracy for it was found on Lacuna, so we reproduce its prompt recipe ourselves.
The external-verifier competitor is the DeBERTa-v3 bidirectional-entailment detector from Zero-Shot Multi-task Hallucination Detection. It reports 0.78 accuracy model-agnostic and 0.61 model-aware on SHROOM, and is the strongest number in the file on that benchmark. We compare against it only where a reference text exists, and report the comparison as-is.
The strongest internal-state competitor is the ICR Probe, run in three ways:
- trained with full supervision on the held-out task, as an upper bound (reported AUROC 0.76–0.84 in its paper);
- trained on all source tasks and applied zero-shot to the held-out task;
- trained on all source tasks, then fine-tuned on the $k$ support examples.
Ablations isolate the key ingredient, which is meta-training the feature map for the few-shot fit:
- same features with a logistic regression or SVM fitted on $k$ examples, no encoder;
- a meta-trained encoder with a prototype (nearest-class-mean) classifier instead of the convex learner;
- each feature family removed in turn.
Fairness. All methods see the identical $k$ support examples, drawn with 20 seeds. All use the same frozen LLM, and all get the same hyperparameter budget, tuned only on held-out meta-validation tasks.
Success criterion. At $k=8$ per class, our probe must beat the source-trained-then-fine-tuned ICR Probe and the no-meta-training SVM by more than seed variance in AUROC. It must also close at least half the gap between zero-shot transfer and the fully supervised in-task ICR Probe.
Why not just…?
Why not just use a zero-shot judge with a good definition? This is a real competitor, and SHROOM-INDElab is the evidence. Against it, HaluEval reports near-chance self-detection (58.53% on summarization), and TruthfulQA's best prompted model reached 58% truthfulness against 94% for humans. Neither of those used definition-heavy prompts, though, so we treat this as an open empirical question. The main table settles it task by task. If the zero-shot judge matches our probe on most held-out tasks, the conclusion is that labels are not worth collecting, and we will report that.
Why not just train one big probe on every source task and use it as is? If internal hallucination signals were task-invariant, few-shot adaptation would be unnecessary. This is the single biggest threat to the project. The ICR Probe paper reports strong in-distribution numbers, but the file gives no held-out-task numbers. Our first experiment therefore measures the transfer gap directly. If zero-shot transfer is within seed noise of in-task supervision, the meta-learning contribution collapses, and we would reframe the work around that finding.
Why not just fit logistic regression on the $k$ examples? With good features, plain small-sample fitting might be enough. The no-meta-training ablation tests exactly this. Our expectation, which is not established, is that raw per-layer features are too high-dimensional and noisy for eight examples, which is the regime where MetaOptNet's learned embedding helped in vision. If plain fitting ties, the contribution reduces to the choice of features, and we would report it as that.
Why not just use an NLI verifier? It is strong where a reference exists, at 0.78 on SHROOM model-agnostic. Its own authors note that it cannot run without a premise. Our probe needs no reference, and we evaluate on open QA and TruthfulQA, where the NLI verifier cannot run. Where both apply, we report results head-to-head.
Decisive Checks
The project has three decisive checks. First, does a transfer gap between source-trained and in-task probes exist on held-out tasks? Without one, the motivation fails. Second, does meta-training beat the plain small-sample probe on the same features, measured by AUROC against $k$ from 1 to 32? Third, does the meta-learned probe beat a calibrated few-shot LLM judge given the same labels? This is the clean test of whether few-shot labels belong in the classifier or in the prompt.
We also report two supporting results. One is the AUROC-versus-$k$ curve for each held-out task. The other is whether gains concentrate on the SHROOM tasks, where the gap to existing systems is widest (0.61 model-aware for NLI).
References
- SHROOM-INDElab at SemEval-2024 Task 6: Zero- and Few-Shot LLM-Based Classification for Hallucination Detection
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation
- ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMs
- One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMs
- Meta-Learning with Differentiable Convex Optimization
- Rectifying Demonstration Shortcut in In-Context Learning
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- Zero-Shot Multi-task Hallucination Detection