TL;DR
This paper introduces FactAlign, a novel black-box method for detecting and classifying factual hallucinations in Large Language Models (LLMs). By transforming the problem into a knowledge graph alignment task, FactAlign achieves high performance without requiring access to internal model probabilities or extensive sampling, making it a practical solution for real-world applications.
What Problem Does This Paper Address?
Large Language Models (LLMs) are incredibly powerful, capable of generating human-like text across various domains. However, a significant hurdle to their widespread adoption is their tendency to "hallucinate" – produce statements that are not factually accurate. This unreliability erodes trust in LLM outputs. Existing methods for detecting these hallucinations often fall into a few categories: they might require access to the LLM's internal token probabilities (which are not always available, e.g., for models like ChatGPT), rely on external databases to verify facts (which can be slow or incomplete), or use sampling techniques that involve generating multiple responses from the LLM, increasing computational cost. This paper aims to address these limitations by proposing a method that is both effective and practical, working as a "black-box" that only needs the generated text as input.
Methods
The core innovation of FactAlign lies in reframing hallucination detection as a knowledge graph (KG) alignment problem. The process involves several key steps:
- Knowledge Graph Construction: Both the source text and the LLM-generated text are converted into knowledge graphs. This involves:
- Coreference Resolution: Identifying and linking mentions that refer to the same entity.
- Named Entity Recognition (NER): Identifying entities like persons, organizations, and locations.
-
Relationship Extraction (RE): Identifying the relationships between these entities. The output is a set of relational triplets (subject, predicate, object).
-
KG Alignment: The constructed KGs from the source and generated texts are then aligned. Instead of aligning individual entities, FactAlign aligns triplets. Each triplet is represented as a piece of text and passed through a transformer model to generate a vector representation. The alignment module then finds the best match for each generated triplet within the source triplets.
- Alignment Score Calculation: An alignment score ($s_a$) is computed for each generated triplet, indicating its factual consistency with the source text. A score above a certain threshold suggests the triplet is factual.
- Hallucination Classification (Knowledge Change Detector - KCD): For triplets deemed potentially hallucinatory (below the alignment threshold), a Knowledge Change Detector (KCD) module is employed. This module uses a Natural Language Inference (NLI) model to compute a contradiction score between the hallucinated triplet and source triplets.
- A high contradiction score indicates intrinsic hallucination (manipulation of information present in the source).
- A low contradiction score suggests extrinsic hallucination (addition of information not inferable from the source).
This approach is unique because it operates as a black-box, doesn't require fine-tuning or multiple LLM samples, and can classify hallucinations into intrinsic and extrinsic types.
Datasets
The paper evaluates FactAlign on two datasets:
- WikiBio GPT-3 Hallucination Dataset: Used for hallucination detection. This dataset contains Wikipedia-like passages generated by GPT-3, with sentences labeled as accurate, minor inaccuracies, or major inaccuracies. For evaluation, major and minor inaccuracies were grouped into a "hallucinated" class.
- XSum Hallucination Annotations Dataset: Used for hallucination classification. This dataset contains articles and summaries, with hallucinated spans annotated as either intrinsic or extrinsic.
Key Insights and Ideas
- KG Alignment as a Powerful Analogy: The paper effectively frames hallucination detection as a problem of aligning structured knowledge representations, which proves to be a robust approach.
- Fact-Level Granularity: By focusing on individual triplets (facts) rather than entire sentences or documents, FactAlign offers a more granular and explainable detection mechanism.
- Black-Box Practicality: The method's ability to function as a black-box, without needing internal model states or extensive sampling, makes it highly applicable to real-world LLM deployments where such access is often restricted.
- Intrinsic vs. Extrinsic Classification: The extension to classify hallucination types provides valuable insights into why a hallucination occurred, aiding in debugging and model improvement.
Key Results
FactAlign demonstrated strong performance on both tasks:
- Hallucination Detection: Achieved an F1-score of 0.889 on the WikiBio dataset. This performance was achieved without any fine-tuning or additional LLM samples, outperforming baselines like SelfCheck and AlignScore in terms of F1-score, particularly in recall.
- Hallucination Classification: Achieved an F1-score of 0.825 on the XSum dataset for classifying hallucinations into intrinsic and extrinsic types.
Limitations and Future Work
The authors acknowledge a few limitations:
- Entity Scope: The current KG construction method is limited to named entities. This might cause it to miss important triplets where entities are not named. Future work could explore more advanced relation extraction techniques to build richer knowledge graphs.
- Scalability: Aligning each generated triplet with the entire source knowledge graph can be computationally challenging for very large KGs. The authors suggest exploring retrieval-augmented generation (RAG) techniques to retrieve only relevant sub-graphs for alignment, which could significantly improve scalability.
Summary updated