Abstract
While Large language models (LLMs) have garnered widespread applications across various domains due to their powerful language understanding and generation capabilities, the detection of non-factual or hallucinatory content generated by LLMs remains scarce. Currently, one significant challenge in hallucination detection is the laborious task of time-consuming and expensive manual annotation of the hallucinatory generation. To address this issue, this paper first introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. Furthermore, we propose a zero-resource and black-box hallucination detection method based on self-contradiction. We conduct experiments towards prevalent open-/closed-source LLMs, achieving superior hallucination detection performance compared to extant baselines. Moreover, our experiments reveal variations in hallucination proportions and types among different models.
TL;DR
AutoHall is a new framework that automates the creation of model-specific hallucination datasets by repurposing existing fact-checking benchmarks. By forcing models to generate their own "supporting evidence" for known facts, the researchers can pinpoint exactly where a model’s internal knowledge fails, while also introducing a highly effective "self-contradiction" method for detecting these lies in real-time.
What Problem Does This Paper Address?
Large Language Models (LLMs) are notorious for "hallucinating"—generating information that sounds confident and plausible but is factually incorrect. To fix this, researchers need datasets of these errors to train and evaluate detection systems.
However, current datasets face three major hurdles:
- Manual Labor: Human annotation is incredibly slow and expensive.
- Model Specificity: A hallucination for GPT-4 might be a factual statement for Llama-3. Datasets need to be tailored to the specific model being tested.
- Time Sensitivity: As models are updated, they "learn" away old hallucinations, making static datasets obsolete almost immediately.
The authors of AutoHall aim to solve these issues by creating a pipeline that automatically generates fresh, model-specific hallucination datasets without requiring human intervention.
Methods
The researchers introduced a two-part solution: AutoHall (the dataset generator) and a Self-Contradiction detection method.
The AutoHall Pipeline
- Reference Generation: The system takes a claim from a public fact-checking dataset (where the ground truth is already known) and asks the LLM to provide a "reference" or summary supporting that claim.
- Claim Classification: The LLM is then asked to judge if the original claim is true or false based only on the reference it just wrote.
- Hallucination Collection: If the LLM’s judgment contradicts the ground truth (e.g., it says a true claim is false because it hallucinated a fake reason), that response is flagged as a hallucination.
The Detection Method
To detect hallucinations in the wild, the authors propose a "black-box" approach. They query the model multiple times using different, semantically similar prompts. If the model provides inconsistent or contradictory information across these queries, it’s a strong signal that the model doesn't actually "know" the fact and is likely hallucinating.
Datasets
The study utilizes three primary public fact-checking datasets to seed the AutoHall process:
- Climate-fever: Claims related to climate change.
- PUBHEALTH: Claims regarding public health topics.
- WICE: Real-world entailment claims derived from Wikipedia.
Using these, they generated the AutoHall dataset, which includes specific hallucinatory and factual references for five major models: ChatGPT (GPT-3.5), GPT-4o, Llama2-7B, Llama2-13B, and Llama3-8B.
Key Insights and Ideas
- The Temperature Paradox: Common wisdom suggests that lowering an LLM's "temperature" (randomness) reduces hallucinations. The authors found this isn't strictly true. If a model’s most probable answer is a hallucination (due to bad training data), lowering the temperature just makes the model lie more consistently.
- Topic Sensitivity: Hallucinations aren't evenly distributed. GPT models are most prone to hallucinating about History, Technology, and Culture, while the Llama series struggles more with Politics and Sports.
- Self-Contradiction as a Proxy for Knowledge: The core "aha moment" is that when a model knows a fact, it is remarkably consistent. When it doesn't, it tends to "confabulate" different details every time it's asked, making inconsistency a highly reliable (and zero-resource) detection signal.
Key Results
- Prevalence: Across all tested models and temperatures, the hallucination rate remained stubbornly between 15% and 30%.
- Model Performance: GPT-4o exhibited the lowest hallucination rates across almost all datasets, confirming that larger, more advanced models generally possess more robust internal knowledge bases.
- Detection Accuracy: Their self-contradiction detection method consistently outperformed existing baselines like SelfCheckGPT and SemEnt. For instance, on the WICE dataset with ChatGPT, their method achieved an F1 score of ~75%, significantly higher than the ~65% achieved by standard prompting methods.
- Efficiency: By using $K=6$ (six comparison pairs), the authors found an optimal balance between detection accuracy and computational cost.
Limitations and Future Work
The authors acknowledge a few key limitations:
- Classification Accuracy: The method relies on the LLM being a "good classifier" of its own references. While they measured this accuracy at 92%, the remaining 8% of errors could introduce noise into the dataset.
- Scope: The paper focuses on factuality hallucinations (external truth). It does not deeply explore faithfulness hallucinations, where a model fails to follow the logic of a provided context even if the facts are correct.
- Future Directions: The authors suggest expanding the framework to handle more complex reasoning tasks and exploring how these automated datasets can be used to "fine-tune" models to be more honest.
Summary updated