Two-tiered Encoder-based Hallucination Detection for Retrieval-Augmented Generation in the Wild

Ilana Zimmerman, Jadin Tredup, Ethan Selfridge, Joseph BradleyEMNLP 2024

TL;DR

This paper introduces RAGHalu, a novel two-tiered encoder-based system for detecting hallucinations in Retrieval-Augmented Generation (RAG) systems, specifically tailored for enterprise customer service. RAGHalu significantly outperforms existing methods by first identifying verifiable statements and then checking their support against retrieved knowledge, while also handling non-verifiable statements.

What Problem Does This Paper Address?

Large Language Models (LLMs) are powerful, but they can "hallucinate" – generate factually incorrect or ungrounded information. Retrieval-Augmented Generation (RAG) aims to mitigate this by providing LLMs with external knowledge. However, even RAG systems can hallucinate, posing a significant challenge for enterprise applications where factual accuracy is paramount. Existing hallucination detection methods often rely on public datasets, don't consider real-world production constraints like latency, or fail to distinguish between verifiable and non-verifiable statements (like small talk). This paper addresses the need for a robust, efficient, and enterprise-ready hallucination detection system for RAG.

Methods

The core of RAGHalu is a two-tiered classification system.

  • Tier 1 (RAGHalu-1): This binary classifier acts as a filter, distinguishing between statements that are factually verifiable (VERIFIABLE) and those that are not (NO-INFO). This is crucial for handling conversational elements like greetings or small talk that don't require factual verification.
  • Tier 2 (RAGHalu-2): This classifier takes the VERIFIABLE statements from Tier 1 and determines whether they are supported or unsupported by the retrieved knowledge articles.

The authors chose to use DeBERTa encoder models for each tier, prioritizing their smaller size and faster inference speeds compared to larger decoder-only LLMs, which is critical for production environments. They also developed a 3-label taxonomy to better distinguish between different types of statements.

Datasets

The authors trained and evaluated RAGHalu using a combination of public and private brand data:

  • Public Datasets: FactScore, HaluEval, TruthfulQA, and Databricks Dolly. They re-annotated subsets of TruthfulQA and Dolly to align with their taxonomy and focus on hallucinations relative to retrieved knowledge.
  • Brand Data: Annotated conversations from four large brands (a bank, broadband provider, credit union, and crypto-currency company) using RAG in production. This data was span-annotated by domain experts.

Key Insights and Ideas

  • The Importance of a Two-Tiered Approach: Separating the identification of verifiable statements from the verification process is key. This allows the system to gracefully handle non-factual conversational elements, which is often overlooked by other methods.
  • Encoder Models for Efficiency: Using smaller, faster DeBERTa encoder models for classification is a practical choice for enterprise RAG systems, offering a good balance between performance and computational cost.
  • Domain-Specific Data Matters: Training on brand-specific data significantly improves performance on that brand's test sets, highlighting the need for tailored hallucination detection models in enterprise settings.
  • KB Quality is Crucial: The paper suggests a strong correlation between low article retrieval match scores (KAI score) and incorrect classifications, implying that improving the RAG system's retrieval mechanism is vital for accurate hallucination detection.

Key Results

  • RAGHalu (specifically the RAGHalu-2 tier) achieved an average UNSUPPORTED F1 score of 0.93 on production brand data, outperforming baselines like prompt-engineered GPT-3.5-turbo and fine-tuned Mistral-7B.
  • The RAGHalu system demonstrated strong generalization across different customer service domains and open-source data.
  • RAGHalu is estimated to be at least 5x less expensive per inference than using ChatGPT for hallucination detection, potentially saving a large telecom brand over $105k annually.
  • Classifying single sentences at a time consistently yielded better performance than classifying multiple sentences simultaneously.

Limitations and Future Work

  • Limitations:
    • The performance of the hallucination detection model is inherently limited by the quality of the retrieved knowledge base (KB) and the retrieval system itself. If the KB is irrelevant, most statements might be flagged.
  • The model was developed specifically for customer service RAG systems and may underperform on other data types like paraphrases or summarizations.
  • The paper notes that "mostly-supported" statements (where most information is correct but minor details are off) and statements with implied but unsupported information are challenging.
  • Future Work:
    • Developing more fine-grained hallucination detection models to distinguish between unsupported and contradicting claims, or to identify action statements that might require API integration.
  • Improving KB chunking to better mitigate hallucinations.
  • Exploring ways to correct or edit hallucinations.

Summary updated

BibTeX
@inproceedings{zimmerman2024two,
  author = {Ilana Zimmerman and Jadin Tredup and Ethan Selfridge and Joseph Bradley},
  title = {{Two-tiered Encoder-based Hallucination Detection for Retrieval-Augmented Generation in the Wild}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
}
Figure