SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection

Elisei Rykov, Yana Shishkina, Kseniia Petrushina, Kseniia Titova, Sergey Petrakov, Alexander PanchenkoNAACL 2024

TL;DR

This paper tackles the critical problem of hallucination detection in large language models (LLMs) by exploring novel methods for generating and utilizing synthetic data. The authors found that carefully crafted synthetic data, particularly for paraphrase generation tasks, significantly improved model performance, leading to competitive results in the SemEval-2024 challenge.

What Problem Does This Paper Address?

Large Language Models (LLMs) are incredibly adept at generating human-like text, but they often suffer from "hallucinations." These are instances where the model generates factually incorrect information (factuality hallucinations) or deviates from the given instructions or context (faithfulness hallucinations). This unreliability is a major hurdle for deploying LLMs in real-world applications where accuracy and trustworthiness are paramount. The SemEval-2024 Task 6 specifically aimed to address this by focusing on hallucination detection across definition modeling, machine translation, and paraphrase generation tasks. A key challenge was the limited availability of labeled data for training models to identify these hallucinations.

Methods

The SmurfCat team explored several strategies to tackle hallucination detection:

  • Synthetic Data Generation: Recognizing the scarcity of labeled data, they focused on generating their own.
  • LLaMA2-7b Adapters: They fine-tuned LLaMA2-7b adapters using a small set of annotated examples and then applied these to unlabeled training data. This approach proved more effective for in-domain data augmentation than initial attempts with GPT-4. They trained specialized adapters for generating either correct or hallucinated outputs for each task.
  • GPT-4 Prompting: They also used GPT-4 with carefully designed prompts to generate paraphrases, specifically instructing it to either include or exclude hallucinations. This involved few-shot learning with illustrative examples and explanations.
  • Data Filtration: Both synthetic datasets underwent rigorous filtering to remove noisy or biased samples. This included filtering by length, removing samples starting with certain words, and using the Mutual Implication Score (MIS) to select high-quality examples.
  • Model Fine-tuning: The generated synthetic data was used to fine-tune models, most notably the E5-Mistral model, which is a strong open-source sentence embedding model.
  • Ensembling: To maximize performance, they combined predictions from multiple independent models. Two ensembling strategies were employed: normalized averaging and voting.
  • Mutual Implication Score (MIS) Refinement: They experimented with improving the existing MIS metric by fine-tuning it on different datasets (like PAWS) and using alternative encoders (like Vectara's).

Datasets

The team utilized a combination of existing and newly generated datasets:

  • Existing Datasets:
    • Quora Question Pairs (QQP): Used for initial training of the MIS metric.
  • PAWS (Paraphrase Adversaries from Word Scrambling): Used for fine-tuning the MIS model.
  • Synthetic Datasets:
    • LLaMA2-7b Generated Data: Created by fine-tuning LLaMA2-7b adapters on existing data and then generating paraphrases for unlabeled samples, with specific prompts for hallucinated and non-hallucinated outputs.
  • GPT-4 Generated Data: Created using GPT-4 with specific prompts to generate paraphrases, aiming to introduce hallucinations.

Key Insights and Ideas

  • Synthetic Data is Key: The paper strongly suggests that generating high-quality synthetic data is a viable and effective strategy for hallucination detection, especially when labeled data is scarce.
  • Task-Specific Data Matters: While general synthetic data generation was explored, the most effective approaches involved creating data tailored to specific tasks (e.g., paraphrase generation for faithfulness hallucinations).
  • Careful Filtering is Crucial: Raw synthetic data can be noisy. Rigorous filtering, using metrics like MIS, is essential to ensure the quality and usefulness of the generated data for training.
  • LLaMA2-7b Adapters Outperform Direct GPT-4 Prompting (for data generation): For their specific data augmentation needs, fine-tuning LLaMA2-7b adapters proved more controllable and effective than direct GPT-4 prompting.
  • Ensembling Boosts Performance: Combining multiple models, especially through voting, significantly improved overall performance, demonstrating the benefit of diverse model predictions.

Key Results

The SmurfCat team achieved commendable results in the SemEval-2024 Task 6:

  • Their premier method achieved 9th place in the model-agnostic track and 17th place in the model-aware track.
  • The Voting ensembling approach was found to be the most effective, surpassing most baselines and their own devised methods.
  • The SFT E5-Mistral model, trained on their synthetic data, showed significant performance enhancements.
  • The MIS model, refined with PAWS data and the Vectara encoder, also showed improvements.
  • Interestingly, synthetic data generated by GPT-4 did not yield as good results as LLaMA2-7b generated data, suggesting potential biases in GPT-4's output for this task.

Limitations and Future Work

  • GPT-4 Data Bias: The authors noted that GPT-4 generated synthetic data might contain inherent biases, leading to suboptimal performance.
  • Task-Specific Data Sufficiency: Relying solely on Definition Modeling (DM) synthetic data was insufficient for other tasks. However, paraphrase generation (PG) synthetic data showed promise for addressing hallucinations across multiple tasks.
  • Future Work Suggestions:
    • Centralized Training: Training a single adapter across multiple tasks could enrich the learning context and increase the training dataset size.
  • Exploring Larger Models: Investigating larger LLMs like Mistral-7b, Mixtral-8x7b, or larger LLaMA variants could reveal more efficient architectures.
  • Improving GPT-4 Prompting: Enhancing few-shot prompts with more examples and detailed explanations could mitigate bias and improve the fidelity of GPT-generated data.
  • Contrastive Fine-tuning: Leveraging adapters to generate both positive and negative samples for specific targets could refine training through contrastive fine-tuning, improving the model's ability to discern nuances.

Summary updated

BibTeX
@inproceedings{rykov2024smurfcat,
  author = {Elisei Rykov and Yana Shishkina and Kseniia Petrushina and Kseniia Titova and Sergey Petrakov and Alexander Panchenko},
  title = {{SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure