AutoHall: Automated Hallucination Dataset Generation for Large Language Models

zouying cao, Yifei Yang, hai zhaoSubmission to ICLR 2024 · Rejected

TL;DR

Manual annotation for LLM hallucination benchmarks is slow, costly, and quickly rendered obsolete by model updates. AutoHall solves this by automatically generating model-specific hallucination datasets from existing fact-checking benchmarks, paired with a zero-resource, black-box detection method that spots hallucinations through pairwise self-contradiction.

What Problem Does This Paper Address?

Large language models (LLMs) frequently generate convincing but factually incorrect statements—known as hallucinations. While detecting these errors is critical for safety and trust, building reliable benchmarks typically demands laborious human annotation.

Furthermore, hallucination benchmarks are inherently model-specific and time-sensitive: an annotation set built for one model loses relevance when evaluating another model or an updated checkpoint. Existing zero-resource detection methods (like SelfCheckGPT) attempt to solve this without external knowledge, but often suffer from false positives and mediocre accuracy.

Methods

The authors propose a two-part framework: AutoHall for dataset creation and a pairwise self-contradiction mechanism for detection.

Fact-Checking Claim (with Ground Truth) 
       │
       ▼
[Step 1] Generate Reference via LLM
       │
       ▼
[Step 2] Claim Classification (Does Reference verify Claim as True/False?)
       │
       ▼
[Step 3] Compare with Ground Truth -> Mismatch = Hallucinatory Reference
  1. AutoHall Dataset Generation:
  2. Reference Generation: Given a verified claim from a public fact-checking dataset, the target LLM is prompted to produce an explanatory reference paragraph.
  3. Claim Classification: The LLM is prompted to classify whether the claim is True or False based solely on its generated reference.
  4. Label Assignment: If the LLM’s classification contradicts the real-world ground truth, the generated reference is labeled as a hallucination ($H$). Equal numbers of factual references ($\bar{H}$) are retained to maintain a balanced dataset.
  5. Black-Box Hallucination Detection:
  6. To check whether a generated response $Y$ contains hallucinations, the LLM is queried $K$ independent times ($K = 13$) to produce alternative references ($Y'_k$).
  7. Instead of computing token probabilities or running $n$-gram/BERTScore checks over all samples at once, the LLM evaluates each pair $\langle Y, Y'_k \rangle$ end-to-end for factual contradictions.
  8. If at least one pairwise contradiction is detected, $Y$ is flagged as hallucinatory.

Datasets

The authors construct AutoHall benchmarks using three established fact-checking datasets:

  • Climate-fever (climate-related claims)
  • Pubhealth (public health and biomedical claims)
  • WICE (Wikipedia-based claims spanning law, art, etc.)

Evaluations were conducted across multiple sampling temperatures ($0.1, 0.5, 0.9$) on ChatGPT (gpt-3.5-turbo) and open-source models Llama-2-7b-chat and Llama-2-13b-chat.

Key Insights and Ideas

  • Repurposing Fact-Checking Ground Truth: Rather than having humans judge whether an LLM's reference is true, the pipeline checks whether the reference leads to the correct verdict on a known claim. LLMs perform well on simple binary NLI (validated at 92% human agreement in Appendix C), making this an effective automated filter.
  • Consistent Hallucination Rates: Across different temperatures and model scales, LLM hallucination rates remained remarkably stable at 20% to 30%.
  • Pairwise Checks Avoid Noise Accumulation: Baselines like SelfCheckGPT aggregate across all sampled references simultaneously, which can misattribute conflicts between sampled sentences as errors in the original text. Directly testing $\langle Y, Y'_k \rangle$ pairs isolates contradictions to the response in question.
  • Model-Dependent Vulnerabilities: Topic analysis revealed distinct weak spots: ChatGPT hallucinated most on history, technology, and culture, whereas Llama-2-chat struggled most with politics, technology, and sports.

Key Results

  • Superior Hallucination Detection: On ChatGPT-generated references, AutoHall's detection method achieved F1 scores of 60.00% to 70.66%, outperforming zero-shot CoT, few-shot CoT, and SelfCheckGPT (1-gram) by 20–30% in F1 and improving accuracy by 8.91% on average over SelfCheckGPT.
  • Conflict Frequency as a Diagnostic: Hallucinatory samples generated significantly more conflicts than factual ones. When the number of pairwise conflicts reached 6 or more out of 13, the source response was almost universally a hallucination.
  • Ablation on Sample Count ($K$): Detection F1 steadily rises as $K$ increases from 1 to 13. However, accuracy peaks around mid-range values because larger $K$ increases the false-positive rate (sampling more texts increases the chance of spurious contradictions).

Limitations and Future Work

  • Binary Claim Scope: The generation pipeline relies on claims with definitive "true/false" or "supported/refuted" labels, omitting nuanced, ambiguous, or multi-hop claims.
  • Inference Cost: Querying the LLM $K=13$ times plus performing 13 pairwise classification calls introduces noticeable inference latency and API cost.
  • Self-Evaluation Bias: The detection mechanism depends on the LLM’s ability to faithfully spot contradictions in its own outputs, which may degrade when dealing with subtler factual misstatements or smaller foundation models.

Summary updated

Abstract

While Large language models (LLMs) have garnered widespread applications across various domains due to their powerful language understanding and generation capabilities, the detection of non-factual or hallucinatory content generated by LLMs remains scarce. Currently, one significant challenge in hallucination detection is the laborious task of time-consuming and expensive manual annotation of the hallucinatory generation. To address this issue, this paper first introduces a method for $\underline{auto}$matically constructing model-specific $\underline{hall}$ucination datasets based on existing fact-checking datasets called $\textbf{AutoHall}$. Furthermore, we propose a zero-resource and black-box hallucination detection method based on self-contradiction. We conduct experiments towards prevalent open-/closed-source LLMs, achieving superior hallucination detection performance compared to extant baselines. Moreover, our experiments reveal variations in hallucination proportions and types among different models.
BibTeX
@misc{cao2023autohall,
  author = {zouying cao and Yifei Yang and hai zhao},
  title = {{AutoHall: Automated Hallucination Dataset Generation for Large Language Models}},
  year = {2023},
  eprint = {2310.00259},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.00259},
}

Loading related content...

Figure