uir-cis at SemEval-2025 Task 3: Detection of Hallucinations in Generated Text

Jia Huang, Shuli Zhao, Yaru Zhao, Tao Chen, Weijia Zhao, Hangui Lin, Yiyang Chen, Binyang LiACL 2025

TL;DR

This paper introduces a novel three-stage method for detecting and localizing hallucinations in text generated by large language models (LLMs). By breaking down text into structured "triples" and comparing them against generated factual references, the approach achieves more precise identification of factual errors than previous methods, crucial for building trustworthy AI.

What Problem Does This Paper Address?

Large language models (LLMs) are increasingly used across various applications, but they have a tendency to "hallucinate" – generate factually incorrect, misleading, or unsubstantiated content. This unreliability poses significant risks, especially in critical domains. While existing methods can detect hallucinations at the response or sentence level, they often struggle to pinpoint the exact location of these errors within the text. SemEval-2025 Task 3, Mu-SHROOM, aims to address this by introducing a more granular task: fine-grained hallucination detection and localization across 14 languages. This paper presents a method designed to meet this challenge.

Methods

The proposed method employs a three-stage pipeline:

  1. Triplet Generation: The input text is first converted into a structured representation using "triples." Each triple consists of a subject, predicate, and object, capturing a core factual assertion within the text. This step leverages high-capability language models like GPT-4 to process passages and extract these factual relationships. The goal is to break down complex sentences into more manageable, atomic units.
  2. Reference Generation: For each extracted triple, a factual reference is generated. This is achieved by using an advanced language model (again, GPT-4 or DeepSeek) that accesses internet information and its internal knowledge to create a factually accurate statement corresponding to the triple. The format of these generated references is constrained to ensure they precisely describe the content of the triples.
  3. Hallucination Detection and Localization: The core of the detection happens here. A language model (specifically, FacebookAI/roberta-large) is used to assess the semantic similarity between the original triples and their generated references. This comparison results in a classification of "Entailment," "Neutral," or "Contradiction." A "Contradiction" classification indicates a potential hallucination. The probability of this classification serves as a soft label. If the probability of contradiction exceeds a threshold (0.5), the sample is flagged as containing hallucinations. Localization then focuses on identifying the precise position of the hallucinated object within the original passage, enabling structured output that pinpoints the error.

Datasets

The paper evaluates its method on the unlabeled test sets provided for SemEval-2025 Task 3 Mu-SHROOM. This task's dataset includes generated text across 14 languages: Modern Standard Arabic, Basque, Catalan, Mandarin Chinese, Czech, English, Farsi, Finnish, French, German, Hindi, Italian, Spanish, and Swedish. The dataset also includes sample and validation sets with soft and hard labels for hallucinated spans, and an unlabeled training set.

Key Insights and Ideas

  • Granularity is Key: The paper strongly advocates for moving beyond sentence-level analysis to a more granular approach using structured triples. This allows for a more precise understanding of factual assertions and their potential inaccuracies.
  • Reference Generation as a Ground Truth Proxy: Generating factual references for extracted triples acts as a crucial step in creating a comparative basis for detecting deviations from factual accuracy.
  • Semantic Similarity for Detection: Leveraging semantic similarity between original triples and generated references is an effective way to identify contradictions, which are strong indicators of hallucinations.
  • Structured Output for Localization: The method's ability to not only detect but also precisely localize hallucinations by identifying the erroneous object within a triple is a significant advancement.

Key Results

The method was evaluated on unlabeled test sets across all languages for SemEval-2025 Task 3. The paper reports varying performance across languages. Notably, the method achieved its best performance on the Catalan test set, with an IoU of 0.4644 and a Spearman Correlation (Cor) of 0.5432. Languages like English and Italian showed stable performance with IoU around 0.4 and Cor around 0.5. However, performance dipped significantly in languages like Farsi and Chinese, with IoU below 0.2, suggesting potential challenges with generalization or linguistic complexity in these low-resource languages.

Limitations and Future Work

The authors acknowledge that the method's detection accuracy decreases in some low-resource languages and those with complex syntactic structures and data. They suggest that for such languages, exploring models like XLM-ROBERTa might be beneficial in the future. The paper also implicitly points to the need for further research into improving generalization across a wider range of linguistic complexities and resource levels.

Summary updated

BibTeX
@inproceedings{huang2025uir,
  author = {Jia Huang and Shuli Zhao and Yaru Zhao and Tao Chen and Weijia Zhao and Hangui Lin and Yiyang Chen and Binyang Li},
  title = {{uir-cis at SemEval-2025 Task 3: Detection of Hallucinations in Generated Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure