Key Decision-Makers in Multi-Agent Debates: Who Holds the Power?

Yan Zheng (郑岩), Qian Zhang, Yan Zheng, Jinyi Liu, Hebin Liang, Lanjun WangarXiv 2025 · DOI · arXiv

Abstract

Multi-Agent Debate (MAD) has emerged as a promising paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs). However, the performance of these systems is often inconsistent. This research identifies a critical yet overlooked factor: the order in which agents participate in the debate. We uncover a pronounced "positional bias," where agents positioned later in a debate round wield disproportionate influence over the final consensus. By strategically placing agents with correct initial viewpoints at the end of the sequence—a strategy termed "Truth Last"—we observe accuracy improvements of up to 22.8%. To operationalize these findings in real-world settings where ground truth is unknown, we introduce Multi-Agent Debate Consistency (MADC). MADC utilizes path consistency to dynamically reorder agents, effectively unlocking significant performance gains across various benchmarks and model architectures without requiring additional training or prompt engineering.

Introduction

The quest to scale reasoning in LLMs has led to the development of Multi-Agent systems. While increasing the number of agents or debate rounds can improve performance, the results are frequently unstable and sometimes inferior to simple single-agent Chain-of-Thought (CoT) prompting. This paper addresses this instability by examining the role of agent allocation strategies. We hypothesize that the sequence of interaction is not neutral; rather, the "last word" carries significant weight in shaping the group's final output.

Methodology: Role Allocation and MADC

The study explores four primary role allocation strategies: Fixed (static order), Random (shuffled order), Truth First (correct agents speak early), and Truth Last (correct agents speak late). The "Truth Last" strategy serves as an upper-bound benchmark for the potential of positional optimization.

To translate these insights into a practical tool, we propose Multi-Agent Debate Consistency (MADC). MADC functions as a plug-in for existing MAD frameworks. It first evaluates the "path consistency" of agents—measuring how frequently independent agents arrive at identical intermediate conclusions. Agents are then dynamically reordered such that those with higher consistency scores (and thus a higher likelihood of being correct) are moved to the end of the debate sequence. This ensures that the most reliable agents act as the final decision-makers.

Overview of the Multi-Agent Debate process and the Fixed vs. Random allocation strategies.

Experimental Setup

The framework was rigorously tested on three benchmarks: MATH500 (mathematics), Logical Deduction (deductive reasoning), and Geometric Shapes (text-to-shape interpretation). Evaluations spanned nine models, including the Qwen2.5 series, GPT-4o-mini, GLM-4, and DeepSeek-R1 Distilled models, ensuring broad applicability across different model scales and architectures.

Analysis of Positional Bias

Our "white-box" experiments confirm a strong positional bias in MAD. As shown in Figure 1, the "Truth Last" strategy consistently outperforms all others, including the "Truth First" approach, which often performs worse than random allocation. This suggests that agents tend to converge toward the opinions expressed by those who speak later in the round.

Accuracy comparison across different role allocation strategies on Logical Deduction and Geometric Shapes tasks.

Furthermore, we observed that ordered allocation strategies like "Truth Last" lead to lower debate entropy, indicating a more efficient and confident path to consensus. Our "Agent Threshold" experiments revealed that correctly positioning just half of the agents at the end of a sequence is often sufficient to reverse an incorrect debate outcome.

Results and Discussion

The impact of strategic role allocation is substantial. For the Qwen-2.5-3B model, the "Truth Last" strategy yielded a 22.8% accuracy boost on Logical Deduction and Geometric Shapes. The practical MADC algorithm also showed impressive gains, such as a +9.6% improvement for DeepSeek-Qwen-7B on Geometric Shapes and a +8.8% boost for Qwen-3B on Logic tasks. Notably, the DeepSeek-R1-Qwen-14B model reached a near-perfect 99.6% accuracy on Logical Deduction when utilizing the "Correct Last" strategy. These results demonstrate that MADC effectively stabilizes and enhances the scaling laws of multi-agent systems.

Conclusion

This research highlights that who speaks when is as important as what is said in multi-agent debates. While MADC provides a robust approximation of optimal ordering, it relies on the majority being initially correct. Future work should address scenarios where the majority of agents are initially wrong, potentially leading consistency-based reordering astray. Further exploration into complex power dynamics and social simulation applications, such as mock courts, remains a promising avenue for future study.

Summary updated

Abstract

Recent studies on LLM agent scaling have highlighted the potential of Multi-Agent Debate (MAD) to enhance reasoning abilities. However, the critical aspect of role allocation strategies remains underexplored. In this study, we demonstrate that allocating roles with differing viewpoints to specific positions significantly impacts MAD's performance in reasoning tasks. Specifically, we find a novel role allocation strategy, "Truth Last", which can improve MAD performance by up to 22% in reasoning tasks. To address the issue of unknown truth in practical applications, we propose the Multi-Agent Debate Consistency (MADC) strategy, which systematically simulates and optimizes its core mechanisms. MADC incorporates path consistency to assess agreement among independent roles, simulating the role with the highest consistency score as the truth. We validated MADC across a range of LLMs (9 models), including the DeepSeek-R1 Distilled Models, on challenging reasoning tasks. MADC consistently demonstrated advanced performance, effectively overcoming MAD's performance bottlenecks and providing a crucial pathway for further improvements in LLM agent scaling.
BibTeX
@misc{2025key,
  author = {YAN ZHENG (郑岩) and Qian Zhang and Yan Zheng and Jinyi Liu and Hebin Liang and Lanjun Wang},
  title = {{Key Decision-Makers in Multi-Agent Debates: Who Holds the Power?}},
  year = {2025},
  eprint = {2511.11040},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.11040},
}
Figure