When several language-model agents debate a question and then vote, an answer backed by five agents looks five times as well supported as one backed by a single agent. That count is misleading whenever the five agents agree because they copied each other. Hallucination propagation in multi-agent systems is largely this failure. One agent asserts something false, its peers adopt the claim under social pressure or because they cannot check it, and the system then reports the claim with high confidence. This note proposes a mediator that does two things. It keeps a small, cheaply updatable record of who believes what and on whose word. It also learns how much to trust each agent, both when the agent reasons independently and when it adopts a peer's claim, using only the feedback a deployed system actually gets: whether the one final answer it chose turned out to be right.
The opportunity is that three facts in the literature fit together but have not been combined. Debate collapses into agreement on wrong answers because agents abandon correct positions to match peers. Statistical uncertainty-propagation models fix part of this, but they rely on agents reporting how much they relied on each other. Restricted epistemic logics can track beliefs about beliefs in polynomial time. The missing piece is learning, from sparse outcome feedback and as agent behavior drifts, whether a given act of adoption is evidence or just an echo. That is a bandit problem: a learning problem where you only see the reward of the action you took. It is also non-stationary, and the bandit literature has a specific remedy for that.
Core Claim
Our claim is that support for an answer should be counted by independent derivation, not by head-count. The weight an adopted claim deserves should be learned per agent from outcome-only feedback, with deliberate forgetting of old evidence. This should cut harmful consensus (agreement on a wrong answer) more than centralizing debate under a judge, more than propagation-aware uncertainty scoring, and more than skipping debate entirely, while keeping the accuracy gains that genuine correction during debate provides. The gain should grow with the number of agents, because the number of possible copying paths grows with it.
Concretely, after each debate the mediator holds, for every agent $i$, its private initial answer and any later answer it adopted, together with the sources it adopted from. It scores each candidate answer $a$ as
$$S_t(a) = \sum_{i} w_{i,t}\,\big[\mathbf{1}\{\text{own}_i = a\} + \beta_{i,t}\,\mathbf{1}\{\text{adopt}_i = a\}\big],$$
where $w_{i,t}$ is agent $i$'s learned reliability at question $t$ and $\beta_{i,t}\in[0,1]$ is how much its adoptions are worth. Adoptions accompanied by a new, checkable justification are recorded separately from bare switches.
The mediator samples an answer from a softmax over $S_t$ and observes only whether that answer was correct. If it was correct (in single-answer tasks), the correctness of every agent is revealed. If it was wrong, only the chosen answer's supporters are revealed to be wrong. Losses are importance-weighted by the probability that they were revealed, as in Exp3, the standard exponential-weights algorithm for bandit feedback. Weights then follow a discounted update, $\log w_{i,t+1} = \gamma \log w_{i,t} - \eta\,\hat\ell_{i,t}$, with $\gamma<1$. The parameter $\beta_{i,t}$ is updated in the same way.
Research Program
The work has three components, and each borrows from a different direction.
Belief record. The record is drawn from epistemic logic, the formal study of reasoning about what agents know and believe. Full dynamic epistemic logic is ruled out on cost grounds. Multi-agent satisfiability is PSPACE-complete (Halpern et al.), and arbitrary announcement logics are undecidable (van Ditmarsch et al.; The Undecidability of Quantified Announcements). We instead use a conjunctive, disjunction-free fragment in the style of Belief Update for Proper Epistemic Knowledge Bases. That paper gives a polynomial-time update, $O(|P|\cdot|Q|\cdot 2d)$ in the sizes of the old and new formula sets $P$ and $Q$ and the nesting depth $d$, at the price of being unable to express disjunctions. We cap nesting at depth two, in the spirit of Depth-bounded epistemic logic, which treats reasoning depth as an explicit, bounded resource. The record therefore holds atoms such as "$i$ believes $p$" and "$i$ believes $p$ because it believes $j$ believes $p$."
The true-belief versus mere-belief distinction of Towards Epistemic-Doxastic Planning with Observation and Revision supplies the split between derived and adopted support. That work stays at depth two and assumes transparent observation, and we inherit both limits deliberately. The no-disjunction limitation actually suits this setting, because debate positions are answer commitments, not hedged disjunctions.
Adoption is detected from behavior, not from self-report. Each agent first answers in isolation. Any later switch toward a peer whose message the agent saw is logged as adoption from that peer. A cheap verifier tags a switch as carrying a "new justification" only if the agent's message introduces a step its source did not state. This avoids the self-reported reliance weights that PropUQ-MAS elicits.
Learner. The learner comes from the bandit literature. Multi-Agent Learning for Iterative Dominance Elimination proves that learners which weight all history equally, the Dual Averaging family, can take exponentially many rounds to drop actions that once looked good. Their discounted Exp3 variant fixes this in self-play. Our setting has the same trap. An agent that was reliable on early topics but degrades on a new domain, or under a sycophancy-inducing persona, would otherwise keep its inflated weight. We reuse their discounting idea, but in a single-learner, trust-weighting problem instead of a game. Their polynomial guarantee does not transfer directly, so we treat $\gamma$ empirically. We do offer a regret bound for the importance-weighted estimator against a piecewise-stationary comparator as a secondary theoretical contribution.
Scaling. We study scaling from $N=3$ to $N=9$ agents, with fully connected and sparse communication. Trust parameters are per agent, not per pair, so their number grows linearly in $N$. The belief record grows as $O(N^2)$ atoms per round at depth two, which stays cheap at these sizes. A per-pair variant is tested as an ablation to see whether source identity matters.
Baselines
The naive reference is plain majority vote after free debate, which is what most multi-agent systems ship. We also run independent self-consistency: the same agents answer in isolation and vote, with the same total token budget.
The strongest structural competitor on harmful consensus is the centralized judge of Peacemaker or Troublemaker. That paper reports that centralization reduced Disagreement Collapse Rate, the fraction of debates where initial disagreement collapses into agreement, from about 81% to 41% on CommonsenseQA. Beating it means a lower collapse rate and a lower Negative Agreement Rate (the rate at which agents abandon correct answers) at matched or better accuracy, with the same agent models, persona mixes, and round counts. The file gives no absolute accuracy for that setup, so we will reproduce it ourselves.
The strongest uncertainty-based competitor is PropUQ-MAS. Its reported relative gains over local uncertainty estimates are +6.10% AUROC and +47.58% Prediction Rejection Ratio, averaged over GSM8K, MedQA, and MBPP-Plus. We will run it on the identical debate transcripts and use its propagated risk both to choose answers and to rank them for rejection. Beating it means higher AUROC and PRR when our score $S_t$ is used as the confidence signal.
On medical deliberation, When Agreement Becomes Unsafe reports harmful consensus on DDXPlus reduced "to near zero, an order-of-magnitude improvement over standard pooling." It gives no exact number, it uses fixed precomputed confusion matrices, and it was tested only with two agents. We compare against it with its confusion matrices estimated offline on a held-out split, which gives it full-information access that our mediator lacks. Matching it while learning online from outcome-only feedback would count as success.
Epistemic Gain, Aleatoric Cost trains agents and is not a drop-in aggregator, so it serves as context, not a matched baseline.
Every method gets the same agents, questions, and debate budget. Every tunable method, ours and the baselines, gets a 20-configuration search on a shared validation split.
Three ablations isolate the key ingredients. Setting $\beta\equiv1$ (head-count with learned trust) isolates provenance. Setting $\gamma=1$ isolates forgetting. Reducing the record to depth one, with switches logged but no "on whose word" atoms, isolates the epistemic structure.
Why not just…?
Why not just skip debate and vote over independent answers? This removes propagation entirely and is the most serious threat. If independent self-consistency matches us, the proposal fails: debate would bring no gains worth protecting. The evidence that debate can help is that weaker agents gain from peer messages in Epistemic Gain, Aleatoric Cost, where accuracy rose from 54.8% to 66.5% by round five. Our learned $\beta_{i,t}$ is exactly the test. If it converges to zero for every agent, debate adds nothing, and we will report that.
Why not just use a central judge? A judge halves collapse (Peacemaker or Troublemaker), but it is one more language model reading the same copied transcript. It cannot tell an echo from an independent derivation unless provenance is shown to it. We therefore also run a judge that is given our belief record. If that judge matches the full mediator, the record matters and the bandit learner does not. That would be a narrower but still useful result.
Why not just learn per-agent reliability with standard weighted majority? This is our $\beta\equiv1$, $\gamma=1$ ablation. Standard weighting cannot discount correlated support. On history-symmetric learners, Multi-Agent Learning for Iterative Dominance Elimination gives formal reason to expect slow recovery after drift. The drift experiment below measures this directly.
Why not just use PropUQ-MAS? It already models propagation along a graph. However, it assumes local and inherited errors are conditionally independent, and it relies on self-reported acceptance weights. Sycophantic adoption violates both assumptions. The head-to-head on identical transcripts settles this question.
Proposed Experiments
Main comparison. We run debates of $N\in\{3,5,7,9\}$ agents on CommonsenseQA and MMLU Pro, the sycophancy benchmarks, on GSM8K and MATH500, and on MedQA and DDXPlus. For DDXPlus we use asymmetric loss and report harmful consensus rate. We report accuracy, collapse rate, Negative Agreement Rate, AUROC, PRR, and tokens used. The decisive prediction is that the gap between our mediator and head-count voting widens with $N$. If the gap is flat in $N$, the provenance story is wrong.
Controlled injection. One agent is prompted to assert a planted false premise, injected at different rounds and positions in sparse topologies. We measure how far the false answer spreads and whether the final answer adopts it. Provenance counting should keep the injected answer's effective support near one agent regardless of how many agents echo it.
Drift. Partway through the question stream, we swap which agent receives a high-sycophancy persona. We plot the rounds needed for $w_{i,t}$ to recover, comparing $\gamma<1$ against $\gamma=1$. The prediction is that, without forgetting, recovery takes far longer and harmful consensus spikes during the lag.
Overhead. We report the mediator's wall-clock cost relative to agent generation, confirming that belief updates in the depth-two fragment are negligible at $N=9$.
References
- A Guide to the Modal Logics of Knowledge and Belief: Preliminary Draft
- Undecidability for arbitrary public announcement logic
- The Undecidability of Quantified Announcements
- Belief Update for Proper Epistemic Knowledge Bases
- Depth-bounded epistemic logic
- Towards Epistemic-Doxastic Planning with Observation and Revision
- Multi-Agent Learning for Iterative Dominance Elimination: Formal Barriers and New Algorithms
- Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debate
- When Agreement Becomes Unsafe: Loss-Aware Energy Control for Diagnostic Deliberation
- Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning