Abstract
Multi-Agent Debate (MAD) has emerged as a promising paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs). However, the performance of these systems is often inconsistent. This research identifies a critical yet overlooked factor: the order in which agents participate in the debate. We uncover a pronounced "positional bias," where agents positioned later in a debate round wield disproportionate influence over the final consensus. By strategically placing agents with correct initial viewpoints at the end of the sequence—a strategy termed "Truth Last"—we observe accuracy improvements of up to 22.8%. To operationalize these findings in real-world settings where ground truth is unknown, we introduce Multi-Agent Debate Consistency (MADC). MADC utilizes path consistency to dynamically reorder agents, effectively unlocking significant performance gains across various benchmarks and model architectures without requiring additional training or prompt engineering.
Introduction
The quest to scale reasoning in LLMs has led to the development of Multi-Agent systems. While increasing the number of agents or debate rounds can improve performance, the results are frequently unstable and sometimes inferior to simple single-agent Chain-of-Thought (CoT) prompting. This paper addresses this instability by examining the role of agent allocation strategies. We hypothesize that the sequence of interaction is not neutral; rather, the "last word" carries significant weight in shaping the group's final output.
Methodology: Role Allocation and MADC
The study explores four primary role allocation strategies: Fixed (static order), Random (shuffled order), Truth First (correct agents speak early), and Truth Last (correct agents speak late). The "Truth Last" strategy serves as an upper-bound benchmark for the potential of positional optimization.
To translate these insights into a practical tool, we propose Multi-Agent Debate Consistency (MADC). MADC functions as a plug-in for existing MAD frameworks. It first evaluates the "path consistency" of agents—measuring how frequently independent agents arrive at identical intermediate conclusions. Agents are then dynamically reordered such that those with higher consistency scores (and thus a higher likelihood of being correct) are moved to the end of the debate sequence. This ensures that the most reliable agents act as the final decision-makers.
Experimental Setup
The framework was rigorously tested on three benchmarks: MATH500 (mathematics), Logical Deduction (deductive reasoning), and Geometric Shapes (text-to-shape interpretation). Evaluations spanned nine models, including the Qwen2.5 series, GPT-4o-mini, GLM-4, and DeepSeek-R1 Distilled models, ensuring broad applicability across different model scales and architectures.
Analysis of Positional Bias
Our "white-box" experiments confirm a strong positional bias in MAD. As shown in Figure 1, the "Truth Last" strategy consistently outperforms all others, including the "Truth First" approach, which often performs worse than random allocation. This suggests that agents tend to converge toward the opinions expressed by those who speak later in the round.
Furthermore, we observed that ordered allocation strategies like "Truth Last" lead to lower debate entropy, indicating a more efficient and confident path to consensus. Our "Agent Threshold" experiments revealed that correctly positioning just half of the agents at the end of a sequence is often sufficient to reverse an incorrect debate outcome.
Results and Discussion
The impact of strategic role allocation is substantial. For the Qwen-2.5-3B model, the "Truth Last" strategy yielded a 22.8% accuracy boost on Logical Deduction and Geometric Shapes. The practical MADC algorithm also showed impressive gains, such as a +9.6% improvement for DeepSeek-Qwen-7B on Geometric Shapes and a +8.8% boost for Qwen-3B on Logic tasks. Notably, the DeepSeek-R1-Qwen-14B model reached a near-perfect 99.6% accuracy on Logical Deduction when utilizing the "Correct Last" strategy. These results demonstrate that MADC effectively stabilizes and enhances the scaling laws of multi-agent systems.
Conclusion
This research highlights that who speaks when is as important as what is said in multi-agent debates. While MADC provides a robust approximation of optimal ordering, it relies on the majority being initially correct. Future work should address scenarios where the majority of agents are initially wrong, potentially leading consistency-based reordering astray. Further exploration into complex power dynamics and social simulation applications, such as mock courts, remains a promising avenue for future study.
Summary updated