When several language-model agents debate a problem, every agent usually sees every peer's answer and every agent may revise. Two recent findings show what this costs. Agents that are already right abandon their answer when a peer offers one that is almost right. Longer debates also drift: agents lose track of the question or agree just to finish. The Delphi method, a decades-old protocol for building consensus among human experts, handles this differently. Experts first answer blind. Then each one sees only a group summary, such as the median and spread. Only the outliers are asked to either move toward the group or write down why they will not.
We propose a Delphi protocol for language-model agents whose answers live in a space with a real distance, such as output grids, numeric estimates or judge scores. The protocol computes how spread out the answers are using only pairwise distances. It then uses that spread to decide which agents are outliers who must revise or justify, and which agents are left alone. It stops when the spread stops shrinking. In two sentences: debate fails because the wrong agents get pressured by the wrong peers. A cheap, robust measure of how far apart the agents are can tell us who should be shown what, as Delphi does with human panels, and when to stop.
Core Claim
The tool from geometry is the sandwich bound of Convex formulations of radius-margin based Support Vector Machines. That paper shows that the radius $R$ of the smallest ball enclosing a point set is pinned between the maximum pairwise distance $R_O$ and $1.366\,R_O$. Optimizing $R_O$ is therefore a convex stand-in for the hard enclosing-ball problem. The authors use this for feature selection in support vector machines. They flag the bound's main weakness, that a single outlier inflates $R_O$, and leave a "soft radius" as open work.
We take the pairwise-distance radius out of classifier training and use it as the coordinator's statistic in a Delphi round. Let $a_1,\dots,a_n$ be the agents' current answers and $\delta$ a task distance, such as Hamming distance on output grids, which is Euclidean on one-hot pixel encodings. We define a trimmed radius $R_O^{(k)}$, the maximum pairwise distance after removing the $k$ agents whose removal shrinks it most. This set of remaining agents is the inlier core. Every agent lying farther than $1.366\,R_O^{(k)}$ from the core's medoid (the core answer closest to all others) is treated as an outlier.
Outliers receive the Delphi treatment. They see the anonymized core medoid, the radius, and anonymized core rationales, and they must either revise or submit a justification that is passed to everyone in the next round. Inliers are not shown peer answers at all. They see only the outliers' justifications, as text without the competing answer attached.
Our specific prediction is that this routing protects correct agents from the plausibility trap. The trap documented by Beyond Correctness: Distance-Based Social Dynamics of Multi-Agent Debate operates through exposure to near-correct peer answers. Under our routing, a near-correct peer sits inside the core next to the correct agent, so neither is asked to move. Stopping when $R_O^{(k)}$ fails to shrink for two consecutive rounds should also cut drift without the 1D distributional assumptions of earlier adaptive stopping rules.
Relation to Prior Work
Ha and Mitchell supply the motivation and the measurements. On ConceptARC they show three things: agents far from the truth revise more often, wrong agents' revisions contract toward the truth, and correct agents defect to near-correct peers. Their work is descriptive; it proposes no control mechanism. It also contains a warning we must address. Identical wrong answers exert stronger conformity than diverse ones, so showing outliers a single medoid could amplify herding. This is why the protocol lets outliers justify rather than forcing them to move, and why we ablate medoid-only feedback.
Voting or Consensus? Decision-Making in Multi-Agent Debate supplies two pieces of method and one constraint. The methods are independent first drafts (All-Agents Drafting) and the finding that consensus protocols filter well on knowledge tasks. The constraint is that consensus collapses on reasoning tasks, falling to 28.4% on MuSR against 54.8% for voting. Their agreement is categorical matching of final answers; ours is continuous and drives routing rather than only the final decision.
Multi-Agent Debate for LLM Judges with Adaptive Stability Detection is the closest stopping rule. It applies a Kolmogorov–Smirnov test to a Beta-Binomial model of scalar judge scores between rounds. It measures a change over time in a one-dimensional distribution, not the spatial spread of the cohort, and it does not change who sees what.
The distributed-systems protocol Delphi: Efficient Asynchronous Approximate Agreement for Distributed Oracles contributes the idea of tolerance-relaxed convex validity: the output should stay within the range of honest inputs. We borrow this as an optional drift guard. If the final answer leaves the ball around the round-one core, we revert to the round-one medoid. Their thin-tail assumption on scalar sensor data does not transfer, which is why we use trimming rather than distributional weighting.
The Human-AI Hybrid Delphi Model shows that small panels saturate quickly and that outlier rationales carry the useful conditional information. Its coordination is done by a human facilitator. Ours is a computed rule, which is what makes it testable at benchmark scale.
Research Program
We will build the coordinator in three parts. The first computes pairwise distances, the trimmed radius and the outlier set; this is cheap for $n\le 9$ agents. The second routes the feedback described above. The third tests the stopping criterion each round.
We will study three answer geometries. ConceptARC is the primary setting, because its grids give exact Hamming and Earth Mover's distances and the plausibility trap was found there. LLMBar judge scores are a one-dimensional case, where the trimmed radius reduces to a trimmed range and the comparison with Hu et al. is direct. MuSR is a categorical guardrail. There, distance is 0 or 1 and the method degenerates to plurality-based outlier routing. Its purpose is to check that we do not inherit the consensus collapse.
The analysis has two parts. The first measures per-agent transition rates (correct to wrong, wrong to correct) as a function of each agent's distance to the core, which directly tests the routing hypothesis. The second checks whether $R_O^{(k)}$ actually decreases across rounds, and how often the untrimmed $R_O$ is set by a single agent. The second check is an empirical test of the outlier-sensitivity concern raised by Do and Kalousis.
Baselines
The naive reference point is a single agent with chain-of-thought at the same model size. Its reported ConceptARC base accuracy in the Ha and Mitchell study is 56.82% for GPT-5 mini and 37.16% for DeepSeek-V3.2.
The first strong competitor is static majority voting, or the medoid of independent round-one drafts. This is the "spend compute on samples, not talk" alternative. On LLMBar, Hu et al. report 77.75% for this baseline with Gemini-2.0-Flash. On MuSR, Kaesberg et al. report 54.8% for voting. We run it with the same number of total model calls as our protocol uses, not the same number of agents. This gives it more samples and makes it a fair compute-matched test.
The second competitor is the strongest existing debate method: Hu et al.'s multi-agent debate with Kolmogorov–Smirnov adaptive stopping. They report 81.83% on LLMBar with Gemini-2.0-Flash, using 4–6 rounds instead of 10. We reimplement it with the same agent count (seven, their reported sweet spot), the same models and the same token cap. Beating it means higher LLMBar accuracy at equal or fewer rounds. On ConceptARC we also include fully connected debate with All-Agents Drafting and fixed three rounds. No reported multi-round debate accuracy for ConceptARC was found on Lacuna, so we will establish that number ourselves.
Three ablations isolate the key ingredient. The first broadcasts the same Delphi summary to every agent, with no routing. The second defines outliers categorically, by disagreement with the plurality, rather than by distance. The third uses the untrimmed $R_O$ instead of $R_O^{(k)}$. If the full method does not beat the first two ablations, the geometry is not doing the work.
Why not just…?
Why not just take the medoid of more independent samples? This is the most serious threat, because the contraction effect means even wrong drafts cluster near the truth. Debate only earns its cost if some agents move from wrong to right without dragging right agents to wrong. The compute-matched medoid baseline settles this. We will report it prominently. If it ties our method on ConceptARC, we will conclude that routing helps only as damage control relative to debate, not as a replacement for sampling.
Why not just use Hu et al.'s stopping rule, or stop when categorical answers stop changing? Stopping addresses wasted rounds but not who is pressured. The plausibility trap happens within a single exposure, as Ha and Mitchell observed even in single-step revisions. We therefore swap stopping rules while holding routing fixed, and swap routing while holding stopping fixed. This separates the two contributions.
Why not just run classical Delphi and show everyone the median and spread? The broadcast ablation is exactly this. Ha and Mitchell's conformity finding predicts it may herd correct agents toward a wrong central answer. We expect it to fall between raw debate and routed feedback; if it matches routed feedback, the routing claim fails.
Decisive Experiments
The central test is a controlled injection on ConceptARC following Ha and Mitchell's design. They generated pools of 20 independent answers per task. From these pools we place one correct agent among peers whose answers are near-correct (small Hamming distance) or distant. We then measure how often the correct agent keeps its answer under raw peer exposure, broadcast Delphi feedback and geometric routing.
The hypothesis is confirmed if routing raises retention against near-correct peers substantially while keeping the rate at which wrong outliers become right, and if overall accuracy beats the compute-matched medoid. It is refuted if retention gains come only from agents refusing to revise at all, which shows up as a collapse in wrong-to-right transitions.
Secondary checks are threefold. On LLMBar, the method should match or exceed 81.83% in no more rounds. On MuSR, it must stay near voting's 54.8% rather than consensus's 28.4%. On ConceptARC, the trimmed radius should predict stopping points that avoid the accuracy decline of longer debates.
References
- Convex formulations of radius-margin based Support Vector Machines
- Beyond Correctness: Distance-Based Social Dynamics of Multi-Agent Debate
- Voting or Consensus? Decision-Making in Multi-Agent Debate
- Multi-Agent Debate for LLM Judges with Adaptive Stability Detection
- Delphi: Efficient Asynchronous Approximate Agreement for Distributed Oracles
- The Human-AI Hybrid Delphi Model: A Structured Framework for Context-Rich, Expert Consensus in Complex Domains