DUTh at SemEval-2024 Task 6: Comparing Pre-trained Models on Sentence Similarity Evaluation for Detecting of Hallucinations and Related Observable Overgeneration Mistakes

Ioanna Iordanidou, Ioannis Maslaris, Avi ArampatzisNAACL 2024

TL;DR

This paper tackles the challenge of detecting "hallucinations" – AI-generated text that's factually incorrect or unrelated to the source – by evaluating pre-trained sentence embedding models. They found that T5, a text-to-text transformer, combined with an ensemble of classifiers, achieved the best performance in identifying these errors, highlighting the effectiveness of sentence similarity as a detection mechanism.

What Problem Does This Paper Address?

The proliferation of large language models (LLMs) has brought about impressive text generation capabilities. However, a significant drawback is the phenomenon of "hallucinations," where the generated text deviates from factual accuracy or the provided source material. This not only erodes user trust but also poses risks in applications requiring factual correctness. While previous research has explored hallucination detection, often focusing on machine translation, there's a need for more robust methods that can generalize across different generation tasks and leverage readily available pre-trained models. The SemEval-2024 Task 6: SHROOM aimed to address this by providing a benchmark for evaluating AI-generated text for hallucinations and related overgeneration mistakes.

Methods

The core of the DUTh team's approach lies in leveraging sentence similarity to detect hallucinations. They hypothesize that if a generated sentence (hypothesis) is semantically distant from its intended source or context, it's likely a hallucination.

Here's a breakdown of their methodology:

  • Sentence Embeddings: They utilized pre-trained sentence transformer models (DistilUSE, LaBSE, and T5) from the HuggingFace library to convert sentences into dense vector representations. These embeddings capture the semantic meaning of the text.
  • Cosine Similarity: The cosine similarity between the embeddings of the "hypothesis" (model's output) and the "context" (reference or source text) was calculated. A lower similarity score suggests a potential hallucination.
  • Synthetic Label Generation: To overcome the scarcity of labeled data, they employed a novel approach of generating synthetic labels for unlabeled datasets. This involved using k-means clustering (with k=2) on the calculated cosine similarity and a probability derived from annotator agreement. For the model-aware track, they also incorporated the one-hot encoded representation of the model used.
  • Ensemble Classification: For the final label prediction, they adopted an ensemble strategy. They combined the outputs of several popular classification algorithms (Logistic Regression, Random Forest, Gradient Boosting, K Nearest Neighbours, XGBoost, and Decision Tree) to make the final prediction. The cosine similarity and the hallucination probability were used as features for these classifiers.

Datasets

The paper utilized datasets provided for SemEval-2024 Task 6: SHROOM. These included:

  • Labeled Datasets: Enriched with additional features, these were used for training and evaluation.
  • Unlabeled Datasets: These were used for their synthetic label generation experiments.
  • Guerreiro et al. (2022) Dataset: This dataset was also used for training, particularly in experiments where synthetic labeling was skipped.

The task encompassed three scenarios: Definition Modeling (DM), Paraphrase Generation (PG), and Machine Translation (MT).

Key Insights and Ideas

  • Sentence Similarity as a Hallucination Detector: The paper strongly supports the idea that measuring the semantic similarity between generated text and its source is a viable and effective method for detecting hallucinations.
  • Pre-trained Models are Powerful: Leveraging existing, powerful pre-trained sentence embedding models significantly simplifies and enhances the hallucination detection process.
  • Synthetic Labeling is a Viable Strategy: The successful generation of synthetic labels for unlabeled data demonstrates a promising way to augment training datasets when labeled data is scarce, achieving a notable 75% accuracy in their label extraction mechanism.
  • Ensembling Boosts Performance: Combining multiple classifiers through ensembling consistently outperformed individual models, leading to more robust and accurate predictions.

Key Results

  • T5 Achieves Top Performance: The T5 model, when used with the ensemble classifier and synthetically labeled data, achieved the highest accuracy scores. In the model-aware track, it reached an accuracy of 0.7553, and in the model-agnostic track, 0.7447.
  • Outperforming Baselines: The DUTh system consistently surpassed the provided baseline scores in both accuracy and Spearman correlation.
  • Ensemble Strategy Dominance: The ensemble strategy consistently outperformed individual algorithms across various scenarios, particularly when using synthetically labeled data.
  • Spearman Correlation Insights: T5 also showed strong performance in Spearman correlation, indicating its ability to align output probabilities with annotator perceptions of overgeneration.

Limitations and Future Work

  • Model Comparison: While they compared DistilUSE, LaBSE, and T5, the authors suggest further exploration of other language models to identify the absolute best performer.
  • Dataset Size and Diversity: Training on more diverse and larger datasets could further improve performance.
  • Fine-tuning LLMs for Embeddings: The paper proposes fine-tuning LLMs to extract enhanced embeddings as a future direction, which could lead to more accurate sentence similarity assessments and improved hallucination detection.
  • Generalizability: While the approach showed promise across different tasks (DM, PG, MT), further validation on a wider range of generation tasks would be beneficial.

Summary updated

BibTeX
@inproceedings{iordanidou2024duth,
  author = {Ioanna Iordanidou and Ioannis Maslaris and Avi Arampatzis},
  title = {{DUTh at SemEval-2024 Task 6: Comparing Pre-trained Models on Sentence Similarity Evaluation for Detecting of Hallucinations and Related Observable Overgeneration Mistakes}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure