Enhancing Multi-Agent Debate System Performance via Confidence Expression

Zijie Lin, Bryan Hooi

TL;DR

This paper introduces ConfMAD, a novel framework for Multi-Agent Debate (MAD) systems that explicitly incorporates confidence scores from LLM agents. By allowing agents to express their certainty, ConfMAD significantly improves debate outcomes, leading to higher accuracy and more reliable consensus compared to existing MAD approaches.

What Problem Does This Paper Address?

Multi-Agent Debate (MAD) systems leverage multiple Large Language Models (LLMs) to simulate human debates, aiming to improve task performance. While LLMs can possess strong reasoning abilities, they often struggle to clearly communicate their advantage during debates due to a lack of confidence expression. This can lead to agents stubbornly sticking to incorrect beliefs or prematurely converging on suboptimal answers, ultimately hindering the overall effectiveness of MAD systems. The paper addresses this by proposing a method for LLMs to explicitly express their confidence levels during debates.

Methods

The core of the ConfMAD framework lies in integrating confidence expression into the debate process. The authors propose two methods for eliciting confidence:

  • LN Confidence (Key Length-Normalized Sequence Probability Confidence): This method calculates confidence based on the probability of the extracted answer tokens, normalized by their length.
  • SV Confidence (Self-Verbalized Confidence): This method directly prompts the LLM to output a confidence score between 0 and 100.

To address potential overconfidence issues, ConfMAD also incorporates calibration schemes (Platt Scaling, Histogram Binning, and Temperature Scaling) to adjust the elicited confidence scores. The debate workflow follows a one-by-one format, where agents iteratively refine their arguments and update the debate history, paying attention to the confidence scores of other agents.

Datasets

The authors evaluated ConfMAD on four benchmarks:

  • BIGGSM: A collection of challenging mathematical computation problems.
  • Big-Bench Hard (BBH): A benchmark designed to test general reasoning across diverse tasks.
  • MMLU (Massive Multitask Language Understanding): Evaluates knowledge across multiple academic subjects.
  • MATH: Focuses on high-school competition-level math problems.

Key Insights and Ideas

  • Confidence as a Communication Signal: The paper highlights that confidence scores are not just a measure of an agent's certainty but a crucial communication signal that can guide the debate towards better outcomes.
  • Beyond Simple Majority Voting: ConfMAD demonstrates that incorporating confidence goes beyond simply selecting the answer of the agent with the highest confidence. It encourages deeper reasoning and self-evaluation, leading to more collaborative interactions.
  • Calibration is Key: While confidence expression is important, calibration methods are essential for aligning confidence scores across different LLMs, making them comparable and reliable for decision-making.
  • The Nuance of Confidence: The paper shows that the role of confidence expression in MAD systems is nuanced. While generally beneficial, the effectiveness depends on the quality of elicitation and calibration, and how it interacts with other factors like disagreement and agent capabilities.

Key Results

  • Improved Accuracy: ConfMAD consistently achieved higher accuracy across multiple benchmarks compared to baseline methods like "No Confidence Debate" (No Conf) and "Interventions" (Inter). For instance, on MMLU, ConfMAD settings like LN+Platt outperformed No Conf by 6%.
  • Enhanced Consensus: Figure 4 shows that ConfMAD significantly increases the rate of cases reaching consensus, particularly on BBH and MMLU datasets.
  • Increased Correct Consensus: Figure 5 demonstrates that ConfMAD leads to more instances of correct consensus.
  • Correction Phenomenon: Debates with confidence substantially increased "correction cases" (where an initially incorrect answer is corrected), by approximately 20% on MMLU.
  • Individual LLM Improvement: Table 2 shows that introducing confidence scores improved individual LLM performance within the MAD system, suggesting it guides agents towards better self-evaluation.
  • Calibration Effectiveness: Platt Scaling was found to be a generally stable and effective calibration method, consistently improving ECE (Expected Calibration Error).

Limitations and Future Work

  • Simple Elicitation Methods: The current confidence elicitation methods are relatively simple and may suffer from instability. More advanced techniques, potentially involving multiple sampling, could be explored, though this needs to be balanced with the computational cost of MAD systems.
  • Generalization of Calibration: The current calibration design is dataset-agnostic, which might limit generalization. Future work could investigate retraining on diverse datasets or exploring more adaptive calibration methods.
  • Dynamic Stopping Criteria: The current debate framework uses a fixed number of rounds. Exploring dynamic stopping criteria could improve efficiency.
  • Alternative Communication Strategies: While the paper focuses on one-by-one debates, exploring broadcast modes or other communication strategies could be beneficial.
  • Step-wise Confidence: Applying confidence scoring to step-wise reasoning within multi-step tasks is an area for future exploration.

Summary updated

Abstract

Generative Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks. Recent research has introduced Multi-Agent Debate (MAD) systems, which leverage multiple LLMs to simulate human debate and thereby improve task performance. However, while some LLMs may possess superior knowledge or reasoning capabilities for specific tasks, they often struggle to clearly communicate this advantage during debates, in part due to a lack of confidence expression. Moreover, inappropriate confidence expression can cause agents in MAD systems to either stubbornly maintain incorrect beliefs or converge prematurely on suboptimal answers, ultimately reducing debate effectiveness and overall system performance. To address these challenges, we propose incorporating confidence expression into MAD systems to allow LLMs to explicitly communicate their confidence levels. To validate this approach, we develop ConfMAD, a MAD framework that integrates confidence expression throughout the debate process. Experimental results demonstrate the effectiveness of our method, and we further analyze how confidence influences debate dynamics, offering insights into the design of confidence-aware MAD systems.
BibTeX
@inproceedings{lin2025enhancing,
  author = {Zijie Lin and Bryan Hooi},
  title = {{Enhancing Multi-Agent Debate System Performance via Confidence Expression}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure