Most hallucination detectors are trained to read a source passage and a model's answer together and output "supported" or "hallucinated." This proposal tests whether such detectors partly succeed by reading the answer alone. In common benchmarks the hallucinated answers were themselves written by a language model, so they may carry their own style: different length, phrasing, or confidence. A detector can learn that style instead of checking the answer against the source.
We propose to flip the detector around, using an idea from generative classification, meaning classifying an input by asking which label makes the observed data most probable rather than predicting the label directly. Given an answer and a candidate label, a sequence-to-sequence model scores how probable the source evidence is. The prior over labels given the answer alone is fixed to be uniform. This removes, by construction, the term through which answer-only shortcuts enter. If the approach works, the detector should lose little accuracy when answer style is decoupled from the label, while ordinary discriminative detectors should lose a lot.
Core Claim
Any detector of the probability of label $Y$ given answer $H$ and evidence $R$ factorizes as $P(Y \mid H, R) \propto P(R \mid Y, H)\,P(Y \mid H)$. The second factor, $P(Y \mid H)$, is exactly what an answer-only classifier learns, and it is where generator-style artifacts live. We train a single encoder-decoder conditioned on a label token and the answer to generate the evidence. We then classify with the log-likelihood ratio
$$s(H,R) = \log P_\theta(R_{\text{align}} \mid Y{=}\text{supported}, H) - \log P_\theta(R_{\text{align}} \mid Y{=}\text{hallucinated}, H),$$
where $P_\theta$ is the label-conditioned generator and $R_{\text{align}}$ is the part of the evidence aligned to the answer's checkable content (entities, numbers, dates). Because $P(Y \mid H)$ is set to uniform, answer style cannot contribute label evidence.
We make two predictions. First, on benchmark splits where answer-only cues are decoupled from the label, this detector's accuracy gap between the standard and hard split will be near zero. Second, matched discriminative detectors will show a large positive gap.
The mechanism comes from A Generative Approach for Mitigating Structural Biases in Natural Language Inference. That paper inverted natural language inference to generate the premise from hypothesis and label, under a uniform prior. On SNLI it shrank the gap between standard and hard test sets from +9.74 for discriminative BART to −2.36. It tested only sentence-pair inference, not hallucination benchmarks built from LLM outputs, and that untested setting is the target here. As far as the research file shows, no one has applied this inversion to hallucination detection.
Why the Inversion Is Not Obviously Safe
The proposal is constrained by a negative result. Generative vs Discriminative? Revisiting the shortcut learning debate in text classification showed that capacity-matched generative text classifiers can be worse than discriminative ones. Pure generative GPT-2 reached near-zero worst-group accuracy on Stylized SST-2 and 0.000 on CivilComments, because modeling every token forces the model to absorb stylistic cues. This contradicts the vision results of Generative Classifiers Avoid Shortcut Solutions, which reported 79.4% worst-group accuracy on Waterbirds against 32.2% for standard training.
Our design responds to this conflict in two ways. The model never generates the answer, so answer style has nowhere to enter the likelihood. Shortcuts could still live in the evidence, so we score only the aligned evidence tokens $R_{\text{align}}$. Tokens irrelevant to the claim should receive nearly identical probability under both labels and cancel in the ratio, because a single shared model produces both terms. Whether the risk Sivaprasad et al. identified simply moves to the evidence side is an empirical question, and we test it directly below.
Research Program
The first step measures whether the shortcut exists at all. We train answer-only classifiers that never see the evidence on HaluEval QA, dialogue, and summarization (HaluEval) and on FACTOID (FACTOID). FACTOID builds its pairs through numeric perturbation, entity swaps, and GPT-4 paraphrasing. If answer-only accuracy sits near chance everywhere, the motivation fails and we stop.
The second step builds hard splits in two ways. The first follows the hard-split convention used by Asael et al. for SNLI: test items that the answer-only classifier gets wrong. The second is an intervention split. We paraphrase both the supported and hallucinated answers with one fixed rewriting model, so that style no longer differs by label. Because this split changes the data rather than filtering it, it is the cleaner test.
The third step builds the detector. We use BART and T5, the backbones Asael et al. used, and fine-tune each on the training splits to generate the evidence from the answer and the label.
How we choose the evidence depends on the task. For QA, the evidence is the question plus the knowledge passage. For summarization, the full document is too long, so we use the three source sentences that best match each summary sentence. We use this per-sentence sliding approach because document length limits generative scoring, as Co-opNet found with its 800-token cap.
We pick the aligned tokens with a named-entity and number tagger together with lexical overlap.
The fourth step is a stress test on the evidence side, modeled on the Stylized SST-2 protocol. We inject a stylistic marker into the evidence that agrees with the label 90% of the time, then measure how far worst-group performance drops. This shows where the "explain everything" failure re-enters the method.
Baselines
Naive references. We include an answer-only classifier and the zero-shot ChatGPT judge from HaluEval. HaluEval reports 58.53% accuracy for that judge on summarization, rising to 61.21% with chain-of-thought prompting. The answer-only classifier's score on the hard split defines chance-adjusted difficulty.
Strongest discriminative competitor. This is a cross-encoder of the same size as our model, discriminative BART or a DeBERTa-class model. It reads the answer and evidence together and is trained on the same data. It receives the same hyperparameter search budget and training steps as the generative model.
On FACTOID, the closest reported system is the multi-task factual-entailment model, with an average score of 0.665 against 0.32 for traditional entailment. Its temporal category reaches only 59%. On SHROOM, OPDAI reports 0.836 accuracy on the model-agnostic track with a fused ensemble of LoRA-tuned models. We report against that number while noting that the compute is not matched. For HaluEval QA, KnowHalu reports more than a 15% improvement over prior systems, but the file gives no absolute accuracy for it on Lacuna. We therefore rerun it ourselves rather than quote a figure.
Debiased discriminative baseline. This is the most serious competitor. We train a product-of-experts model, which combines the full detector with the answer-only classifier during training so the full detector cannot profit from answer-only cues. It uses the same backbone and budget as our method.
Ablations of the key ingredients. The first ablation keeps the generative model but adds back a learned $P(Y \mid H)$ instead of the uniform prior. If the uniform prior is what removes the shortcut, this variant should reintroduce the gap. The second scores all evidence tokens rather than only $R_{\text{align}}$, to isolate the alignment choice.
What counts as winning. The method beats the baselines if it shows a smaller standard-to-hard gap than both the cross-encoder and the debiased baseline. It must also be at least as accurate on the intervention split. In-distribution accuracy may be somewhat lower; Asael et al. observed such a cost.
Why not just…?
Why not just debias a discriminative detector with an answer-only expert? This is a real threat, because product-of-experts targets the same $P(Y \mid H)$ term. The difference is that debiasing only down-weights cues that the answer-only expert manages to find. Fixing the prior to uniform removes the term exactly, with nothing to fit. If the debiased baseline matches us on the intervention split, the honest conclusion is that the inversion offers no advantage beyond calibration. We would then report the calibration comparison and treat the result as negative.
Why not just score the answer's likelihood given the source with an off-the-shelf LM? A pointwise-mutual-information score, meaning how much the source raises the probability of the answer, needs no training. However, it scores the answer itself, so it inherits answer fluency effects. TruthfulQA shows that fluent, familiar falsehoods receive high likelihood. Liu et al. report perplexity at about 63% AUROC on CSQA, against about 69% for their geometric scores. We run this score on all splits as a zero-training reference.
Why not just ask a strong LLM judge? HaluEval finds ChatGPT near chance on summarization, and chain-of-thought prompting hurts on QA and dialogue. KnowHalu reports that GPT-3.5 behaves "lazily" when used as a judge. The judge is nonetheless included on the intervention split, since a judge never trained on the benchmark might be immune to its artifacts.
Why not just rebalance the training data? Paraphrase-balanced training is itself an intervention. We train the cross-encoder on paraphrase-equalized data as an additional baseline. If that alone closes the gap, the inversion is unnecessary wherever paraphrasing at scale is affordable.
Decisive Checks
The program is worth pursuing only if three things hold. Answer-only accuracy must be clearly above chance on at least one of HaluEval or FACTOID. The cross-encoder's gap must then be clearly positive while the generative detector's gap stays near zero. That near-zero gap must also not be matched by product-of-experts or rebalancing.
The evidence-side injection test measures the remaining risk. A collapse in worst-group performance there would show that the vulnerability Sivaprasad et al. identified has moved rather than disappeared. Finally, the ablation that adds back a learned $P(Y \mid H)$ should reintroduce the gap. If it does not, the uniform prior is not the reason the method works.
References
- A Generative Approach for Mitigating Structural Biases in Natural Language Inference
- Generative vs Discriminative? Revisiting the shortcut learning debate in text classification
- Generative Classifiers Avoid Shortcut Solutions
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- FACTOID: FACtual enTailment fOr hallucInation Detection
- Cooperative Generator-Discriminator Networks for Abstractive Summarization with Narrative Flow
- OPDAI at SemEval-2024 Task 6: Small LLMs can Accelerate Hallucination Detection with Weakly Supervised Data
- KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- From Out-of-Distribution Detection to Hallucination Detection: A Geometric View