FACTOID: FACtual enTailment fOr hallucInation Detection

Vipula Rawte, S. M. Towhidul Islam Tonmoy, Krishnav Rajbangshi, Shravani Nag, Aman Chadha, Amit P. Sheth, Amitava Das

Abstract

Large Language Models (LLMs) frequently generate factually incorrect content, a phenomenon known as hallucination. While Retrieval-Augmented Generation (RAG) provides a grounding mechanism, existing Textual Entailment (TE) methods for verification lack the granularity to identify specific errors. We present FACTOID, a framework and benchmark for Factual Entailment (FE) that utilizes multi-task learning to detect, locate, and categorize hallucinations. FACTOID achieves a significant accuracy improvement over traditional TE baselines and introduces the Hallucination Vulnerability Index (HVI) for ranking LLM reliability.

Introduction

The rise of Large Language Models (LLMs) has been accompanied by the persistent challenge of hallucinations—confident but false statements. Retrieval-Augmented Generation (RAG) attempts to mitigate this by providing external documents as context. However, ensuring the model's output strictly adheres to these documents requires robust verification. Traditional Textual Entailment (TE) models, typically trained on datasets like SNLI or MNLI, are often insufficient for this task because they lack granularity and context sensitivity regarding specific entities like dates, names, and locations.

The FACTOID Framework

To address these limitations, we propose Factual Entailment (FE), a new paradigm that moves beyond simple binary or ternary classification. The FACTOID framework employs a Multi-Task Learning (MTL) approach to perform three concurrent tasks:

  1. Entailment Classification: Determining if a source document supports or refutes the generated text.
  2. Span Detection: Pinpointing the exact words or phrases that constitute the factual error.
  3. Hallucination Classification: Categorizing the error into specific types, such as "Bothersome Numbers" or "Imaginary Figures."
Overview of the FACTOID framework for Factual Entailment detection.

Technically, the system utilizes high-dimensional embeddings (e.g., e5-mistral-7b-instruct) to maintain context across long documents. It integrates SpanBERT for precise localization and RoFormer to manage long-distance dependencies. To mitigate class imbalance, Dice Loss is used for the entailment task.

The FACTOID Benchmark

We introduce the FACTOID benchmark, a synthetic dataset comprising 2 million text pairs. This dataset extends the HiLT corpus by using GPT-3 and Word2Vec-based entity substitution to create negative samples across four primary hallucination categories:

  • Bothersome Numbers: Incorrect dates, prices, or percentages.
  • Temporal Issues: Chronological inconsistencies.
  • Imaginary Figures: Fictional or misplaced individuals.
  • Places: Erroneous geographical locations.
Examples of 'Bothersome Number' hallucinations in the FACTOID benchmark.

Key Results and Insights

The FACTOID framework demonstrates a substantial performance leap, achieving an average accuracy of 0.665, compared to 0.32 for traditional TE baselines. The model is particularly adept at identifying "Places" (0.71) and "Imaginary Figures" (0.69).

A significant contribution of this work is the Auto Hallucination Vulnerability Index (HVI), an automated metric for ranking LLMs based on their grounding performance. Our evaluation of 15 popular models revealed that Falcon 7B was the most vulnerable to hallucinations, while T0 11B and T5 showed higher levels of grounding. Interestingly, the data suggests that larger models without Reinforcement Learning from Human Feedback (RLHF) may be more prone to certain hallucinations than smaller, specialized models.

Conclusion and Future Work

While FACTOID significantly improves hallucination detection, "Temporal Issues" remain a complex challenge, requiring more sophisticated reasoning about timelines. Future research will focus on enhancing temporal logic and expanding the FACTOID taxonomy to include more nuanced logical fallacies and "soft" hallucinations.

Summary updated

BibTeX
@misc{rawte2024factoid,
  author = {Vipula Rawte and S. M. Towhidul Islam Tonmoy and Krishnav Rajbangshi and Shravani Nag and Aman Chadha and Amit P. Sheth and Amitava Das},
  title = {{FACTOID: FACtual enTailment fOr hallucInation Detection}},
  year = {2024},
  eprint = {2403.19113},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.19113},
}
Figure