A Debate-Driven Experiment on LLM Hallucinations and Accuracy

Ray Li, Tanishka Bagade, Kevin Martinez, Flora Yasmin, Grant Ayala, Michael Lam, Kevin ZhuarXiv 2024 · DOI · arXiv

Introduction

Large Language Models (LLMs) frequently exhibit "hallucinations," generating factually incorrect information with high confidence. While traditional mitigation strategies focus on internal model adjustments—such as fine-tuning or external fact-checking—this research investigates a novel approach: inter-model dynamics. By simulating a social environment where truth and deception coexist, the study explores whether a "debate-driven" framework can reduce collective hallucination and sharpen model reasoning.

Methodology: The Multi-Agent Debate Forum

The researchers established a multi-agent debate forum utilizing multiple instances of GPT-4o-Mini. The framework assigns agents into three distinct roles to test adversarial robustness:

  1. The Saboteur: Instructed to provide false but highly plausible answers.
  2. Fact-Based Models: Instructed to provide accurate answers while remaining aware that misinformation may be present in the group.
  3. The Moderator: A fact-based model that observes two rounds of debate to deduce the final, objective answer.
Overview of the multi-agent debate framework, illustrating the interaction between Fact-Based models, a Saboteur, and a Moderator.

This setup allows for the testing of various model ratios (e.g., 5:1, 4:1, 3:1) to determine how the weight of misinformation influences the final consensus.

Experimental Setup and Datasets

The experiment utilized the TruthfulQA benchmark, a dataset specifically designed to mimic common human falsehoods and misconceptions. Performance was evaluated across several categories, including Law, Health, and Politics. The primary metric for success was "Overall Percentage Accuracy," comparing the Moderator's final choice against the baseline performance of a single, solitary model.

Key Results and Analysis

The results demonstrate that adversarial interaction can serve as a catalyst for accuracy. The inclusion of a debate round involving a saboteur raised the average accuracy from a baseline of 61.94% to a peak of 78.93%—a nearly 17% improvement.

Comparison of overall accuracy across different model ratios versus the baseline performance.

Interestingly, the data suggests diminishing returns on group size; the 3/1 ratio performed almost as effectively as the 5/1 ratio. This indicates that even a small majority of truthful models can effectively neutralize a single source of misinformation.

Domain Sensitivity and Insights

The effectiveness of the debate format varied significantly by domain:

  • High Performance: Fact-heavy categories like Conspiracies and Misquotations maintained high accuracy (86-89%), as the concrete nature of the truth made it easier to defend.
  • Vulnerabilities: Subjective or ambiguous categories, such as Superstitions and Paranormal, were more susceptible to the Saboteur’s influence. Notably, Psychology saw a significant struggle, with accuracy dropping as low as 16.67%.

These findings suggest that LLM accuracy is not merely a function of data retrieval but of the model's ability to reason through conflicting information. The multi-agent setup acts as a form of "collaborative filtering," forcing truthful models to better justify their reasoning when confronted with plausible lies.

Conclusion and Future Work

This study highlights the potential of adversarial debate to mitigate LLM hallucinations. However, limitations remain, including the use of a fixed saboteur strategy and model homogeneity. Future research should explore more diverse model "zoos" and more sophisticated sabotage techniques, such as the use of logical fallacies or gaslighting, to further test the limits of model-driven truth-seeking.

Summary updated

Abstract

Large language models (LLMs) have achieved a degree of success in generating coherent and contextually relevant text, yet they remain prone to a significant challenge known as hallucination: producing information that is not substantiated by the input or external knowledge. Previous efforts to mitigate hallucinations have focused on techniques such as fine-tuning models on high-quality datasets, incorporating fact-checking mechanisms, and developing adversarial training methods. While these approaches have shown some promise, they often address the issue at the level of individual model outputs, leaving unexplored the effects of inter-model interactions on hallucination. This study investigates the phenomenon of hallucination in LLMs through a novel experimental framework where multiple instances of GPT-4o-Mini models engage in a debate-like interaction prompted with questions from the TruthfulQA dataset. One model is deliberately instructed to generate plausible but false answers while the other models are asked to respond truthfully. The experiment is designed to assess whether the introduction of misinformation by one model can challenge the truthful majority to better justify their reasoning, improving performance on the TruthfulQA benchmark. The findings suggest that inter-model interactions can offer valuable insights into improving the accuracy and robustness of LLM outputs, complementing existing mitigation strategies.
BibTeX
@misc{li2024debate,
  author = {Ray Li and Tanishka Bagade and Kevin Martinez and Flora Yasmin and Grant Ayala and Michael Lam and Kevin Zhu},
  title = {{A Debate-Driven Experiment on LLM Hallucinations and Accuracy}},
  year = {2024},
  eprint = {2410.19485},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.19485},
}
Figure