Byun at SemEval-2024 Task 6: Text Classification on Hallucinating Text with Simple Data Augmentation

Cheolyeon ByunNAACL 2024

TL;DR

This paper tackles the challenge of identifying "hallucinated" text generated by AI models – text that is grammatically correct but factually or contextually incorrect. The authors explored various classification methods, finding that while BERT-based models showed initial promise, simple data augmentation was crucial for improving performance on an imbalanced dataset.

What Problem Does This Paper Address?

Large Language Models (LLMs) are becoming increasingly sophisticated, but they still suffer from a phenomenon called "hallucination." This refers to AI-generated text that, while often fluent and grammatically sound, deviates significantly from the user's input or factual reality. This is a critical issue for applications where accuracy and trustworthiness are paramount, such as in information retrieval or content generation. The SemEval-2024 Task 6 (SHROOM) aims to develop methods for automatically detecting these hallucinations, and this paper presents an approach to this classification task.

Methods

The authors experimented with several text classification techniques. Their primary approach involved fine-tuning transformer-based models like BERT and RoBERTa. They also explored simpler methods such as logistic regression, Support Vector Machines (SVC), and Naive Bayes, using word embeddings derived from BERT. A key aspect of their methodology was data augmentation, specifically replacing words with synonyms in a subset of the data, to combat the issue of limited and imbalanced training data. They also investigated concatenating input features ("src", "data", "tgt") with prefixes to guide the model's attention mechanism.

Datasets

The primary dataset used is from SemEval-2024 Task 6 (SHROOM). This dataset consists of text examples annotated by five annotators, with a majority vote determining the final label (hallucinating or not hallucinating). The paper notes that the validation dataset was imbalanced, with more non-hallucinating examples than hallucinating ones.

Key Insights and Ideas

  • Data Augmentation is Crucial for Imbalanced Datasets: The paper highlights that even with powerful models like BERT, performance can be significantly hampered by imbalanced training data. Simple data augmentation, even if restricted, proved vital for improving accuracy.
  • Transformer Models Show Promise, but Need Careful Tuning: BERT and RoBERTa demonstrated higher initial accuracy compared to traditional methods, suggesting their suitability for this task. However, the gap between initial training/validation performance and the final submitted score indicates challenges in generalization and overfitting.
  • Feature Concatenation Can Be Effective: Concatenating input features with prefixes ("src", "data", "tgt") simplified data handling and allowed the attention mechanism of transformer models to effectively learn relationships without significant loss in accuracy.
  • The "Model" Feature Was Not Utilized: The paper found that the "model" column in the dataset did not contribute to improved accuracy, suggesting it was either redundant or not informative for the classification task in their setup.

Key Results

The initial attempt using BERT achieved an accuracy of 80% on a simple train-test split. However, the final submitted score to SemEval was only 60%. RoBERTa achieved 76%, while SVC, Naive Bayes, and Logistic Regression performed significantly lower, with accuracies of 50%, 48%, and 46% respectively. The Spearman correlation coefficient for the expected train and test data from BERT was 0.64, but the actual submission's Spearman correlation was a lower 0.23, indicating a significant drop in performance and room for improvement compared to top teams.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Cross-Validation: The use of a simple train-test split instead of cross-validation is identified as a limitation. Implementing cross-validation could provide a more robust estimate of the model's true accuracy on unseen data.
  • Overfitting: The significant drop in accuracy from initial experiments to the final submission suggests overfitting. The authors propose using techniques like Lasso regression to penalize high variance.
  • Ensemble Learning: They suggest that a more holistic ensemble learning approach, potentially separating data based on tasks and using multiple models, could lead to better generalization and higher accuracy.
  • Data Augmentation Strategies: The authors believe a more thorough and systematic approach to data augmentation, exploring various techniques beyond synonym replacement and scaling up the data significantly, could further boost performance. They also suggest utilizing the probability feature of the input dataset more effectively within an ensemble pipeline.

Summary updated

BibTeX
@inproceedings{byun2024byun,
  author = {Cheolyeon Byun},
  title = {{Byun at SemEval-2024 Task 6: Text Classification on Hallucinating Text with Simple Data Augmentation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure