Debating on the Relevant Subgraph: Live Argument Graphs as the Memory of Multi-Agent LLM Debate

When several language models debate a question, each one usually reads the entire growing transcript before it speaks. That habit is a likely root of debate's best-documented failures. A weak but stubborn model can drag a stronger one below what the stronger model scores alone. Agents give in to persistent opponents rather than to good arguments. Reasoning models spend extra tokens building justifications for false premises instead of rejecting them. We propose to replace the transcript with a live argument graph. Each contribution becomes a node, and each rebuttal becomes a directed "attack" edge. An exact symbolic procedure keeps every argument labeled as currently accepted, defeated, or undecided. Before a turn, an agent sees only the small piece of the graph that connects the point it is answering to the claim under dispute, and that piece arrives with each argument's current status attached.

In two sentences: debate fails partly because agents reason over everything that has been said, including arguments that are already defeated or irrelevant. We maintain the debate as a formally evaluated argument graph and show each agent only the local subgraph through which its next move can change the verdict. The bet is that this makes debate robust to distraction and to rationalization.

Core Claim

Our central claim is that the right context for a debate turn is a dialectical enclosing subgraph. This idea borrows from knowledge-graph link prediction, where the context for judging a link is the small subgraph lying on short paths between its two endpoints. Applied to debate, the endpoints are the newly added argument and the root claim. The argument-dynamics literature gives this choice a principled justification. Under semantics that satisfy directionality, an update can only change the status of arguments downstream of it. So an argument can affect the root claim only if it lies on an attack path toward that claim.

Concretely, let $G_t=(A_t,R_t)$ be the argument graph after turn $t$, where $A_t$ is the set of arguments and $R_t$ the set of attack edges. Let $c$ be the root claim and $a$ the argument just added. The context for the next turn is

$$ S_k(a,c)=\{x\in A_t : d(a,x)+d(x,c)\le k\}\;\cup\;\{y : (y,x)\in R_t,\ x \text{ in the first set}\}, $$

Here $d(\cdot,\cdot)$ is directed attack-path length and $k$ is a hop budget. The second set is the "conditioning boundary," meaning the outside attackers of on-path arguments. Every node in $S_k$ is shown to the agent with three things: its natural-language text, its current grounded status, and its pair of distances $(d(a,x), d(x,c))$.

The status labels are always computed exactly over the whole graph, using incremental evaluation. Only the agent's view is local. The verdict is the label of the root claim. If that label is undecided, a judge model decides.

What would be new is the combination of three things none of the prior systems has together. The debate state is a dynamically updated formal argument graph rather than a text dump. The verdict comes from a solver rather than from consensus or a judge's reading of a transcript. Each agent's context is chosen by path-based relevance to the disputed claim rather than by recency or summarization.

Where This Sits

Three lines of work motivate the gap. Examining Inter-Consistency of Large Language Models Collaboration introduces FORD. In FORD, agents pick stances, take turns refuting or conceding, and a GPT-4 judge summarizes. The same paper documents the distraction effect, in which mismatched debates pull a stronger model below its solo accuracy. A Diagnostic Study of Multi-Agent LLMs for Real-World Debates shows that debate consensus barely beats chance at predicting validity, because it hides sycophantic capitulation. It also supplies process metrics such as responsiveness and influence asymmetry, which we reuse. Thinking in a Crowd shows that "thinking" modes rationalize misleading auxiliary context and end up below non-thinking accuracy.

All three treat the dialogue as unstructured text. None gives the model a way to know that a premise has already been defeated. Step-level debate in ChainLM and the skeptic-versus-truster game in the Markov chain-based debate framework add structure to the protocol, but not to the state the agents condition on.

Several papers supply the method. Efficient Computation of Extensions for Dynamic Abstract Argumentation Frameworks recomputes extensions on a reduced framework built from the arguments an update can influence, and reports speedups of up to two orders of magnitude. Dynamics of argumentation systems: A division-based method proves that local recomputation is sound when the unaffected part is fixed and its attacks into the affected part are kept as a conditioning component. Our boundary set is exactly that component. Inductive Relation Prediction by Subgraph Reasoning (GraIL) contributes the idea of an enclosing subgraph between two target nodes and the double-radius distance labels. GraIL feeds these to a graph neural network. We hand them to a language model as readable positional cues, and we apply them to a defeasible graph rather than the static factual graph GraIL assumes.

Other papers constrain the design. Change in Abstract Argumentation Frameworks: Adding an Argument shows that one new argument can have non-local ripple effects. This is why we never let the agent's local view determine statuses; the exact global solver does. The reduced-framework approach is known to miss stable extensions in some updates. We therefore use grounded semantics, which is unique and polynomial, as the primary verdict, and report preferred semantics only as a secondary analysis. Formal Analysis of Dialogues on Infinite Argumentation Frameworks shows that protocol and strategy can change outcomes on identical graphs. This is a warning that our protocol choices are themselves variables to report.

The closest neural-symbolic bridge is Enhancing conflict resolution in language models via abstract argumentation together with its benchmark study. These papers fine-tune models to compute grounded labels on pre-extracted static graphs of 6 to 25 arguments. They report 0.54 zero-shot accuracy for GPT-4o, and 0.97 accuracy with full conflict-freeness after supervised fine-tuning on solver-style traces. Those papers assume the graph is given, keep it static, and use one model. Our setting is the opposite: the graph is built from live natural-language debate, it changes every turn, and the labels come from a solver rather than a model.

SPLIT-RAG resolves conflicts among parallel retrieval agents by finding a maximum-weight clique of mutually compatible facts. That approach is symmetric and directionless. Argumentation semantics instead distinguishes "A defeats B" from "B defeats A" and propagates reinstatement, where defeating an attacker restores the argument it attacked. Whether that directionality matters in practice is something we test.

Research Program

The system has four components.

The first is a turn protocol. An agent is assigned a stance. It is shown the enclosing subgraph around the last update together with status labels. It must name a target argument, which is either one currently accepted against its stance or one currently defeating its own side, and then write a single new argument.

The second is an edge extractor. A separate, cheap model call decides which arguments inside a candidate window the new argument attacks. This is the hard grounding problem, and we restrict it to local pairwise judgments. The candidate window is the target plus the nodes in its enclosing subgraph, so the extractor never scans the whole graph.

The third is an incremental grounded-semantics solver in the style of Alfano et al. It updates labels globally after each turn. The paper by Cerutti et al. notes that competition solvers are built for static graphs, which is why we implement incremental evaluation directly.

The fourth is a verdict rule: the stance whose root claim is labeled accepted wins, and an undecided root goes to a judge.

We will study three questions. The first is whether status-labeled local context reduces distraction and capitulation. The second is whether the subgraph restriction adds value beyond showing the labeled graph in full. The third is how badly edge-extraction errors propagate through exact semantics. A single wrong attack edge can flip the root, so this sensitivity is the main technical risk. We will measure edge accuracy against human annotation on a sample. We will also inject controlled edge noise to plot verdict accuracy as a function of edge error.

Baselines

The naive reference point is the strongest single model answering alone with chain-of-thought. This is essential because the distraction effect is defined against it. Alongside it we run self-consistency, meaning majority vote over independent samples, at a token budget matched to the full debate. Matching the budget ensures that any gain is not just extra compute.

The strongest existing debate method is FORD, run exactly as described: stance selection, alternating refute-or-concede rounds, and a judge. We use the same agent models, the same number of rounds, the same judge model when one is needed, and the same total token budget, and we give the same prompt-tuning effort to both systems on a shared development split. The file contains no reported FORD numbers beyond the qualitative findings (no reported result found on Lacuna for FORD accuracy on αNLI, CSQA, COPA, e-CARE, Social IQa, PIQA, or StrategyQA). We will therefore reproduce FORD ourselves. Beating it means two things. First, higher accuracy on these seven benchmarks. Second, and more importantly, in mismatched strong–weak pairings our team accuracy must stay at or above the strong model's solo accuracy while FORD falls below it.

For the knowledge-graph conflict setting, the strongest competitor is SPLIT-RAG's maximum-weight clique filtering. It reports Hits@1 of 85.3% on WebQSP and 63.0% on CWQ. We will rerun it with our backbone models rather than compare to the reported numbers directly. Beating it means matching its Hits@1 on clean questions while doing better on questions where we inject contradictory facts with a known direction of defeat.

Three ablations isolate the key ingredient. The first gives agents the full transcript plus solver labels, which tests whether locality matters. The second gives the full labeled graph instead of the enclosing subgraph, which separates structure from pruning. The third gives the enclosing subgraph with labels removed, which tests whether status information matters. A fourth variant replaces the solver with a model that reads the graph and assigns labels itself. This tests whether exact semantics matters. Running it with a model fine-tuned in the manner of Li et al. makes that comparison fair to the strongest neural alternative.

Why not just…?

Why not just summarize the transcript to shorten context? Summarization shortens context but does not mark which claims have been defeated. The rationalization trap documented in Thinking in a Crowd is about models working to reconcile false premises they are shown, not about context length as such. A summary can preserve a defeated but persuasive premise as easily as a sound one. We include a length-matched summarization baseline. If it equals the enclosing-subgraph condition on the misleading-premise tasks, our claim that dialectical relevance matters is wrong.

Why not just let a strong judge read the full debate, or let a fine-tuned model compute the labels? The judge route is what FORD already does, and the diagnostic study by Pitre et al. shows that outcome-level judgments hide capitulation. The fine-tuned labeler is a real threat on small graphs, since it reaches 0.97 grounded accuracy up to 25 arguments. However, that result covers clean, static graphs, while the solver is exact at any size for negligible cost. The model-as-labeler ablation settles this question. If it matches the solver on our debate graphs, which we expect to stay under about 35 nodes, then the solver is a convenience rather than a contribution, and we will say so.

Why not just show the whole labeled graph and skip subgraph extraction? This is the most serious alternative. With short debates the whole graph may be small enough that pruning adds nothing. The full-graph ablation tests it directly, and we predict the gap will grow with debate length and with the number of off-path arguments. To test that prediction, we vary the number of rounds and add irrelevant but true side arguments. The redundant-information trap in Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? suggests off-path material will hurt. If the gap stays flat, we will report locality as unnecessary and keep the labeled graph as the contribution.

Proposed Experiments

The first experiment tests the distraction effect. We run the seven FORD commonsense benchmarks with matched and mismatched model pairs, such as a strong and a weak open model. For each condition we report accuracy, the gap between team and strong-solo accuracy, and Pitre et al.'s responsiveness and influence-asymmetry metrics. The decisive check is whether our system eliminates the negative team-minus-solo gap that FORD shows.

The second experiment tests premise injection. One agent, or the retrieved context, is seeded with a plausible false premise, following the auxiliary-misinformation setup of Zhao et al. We measure how often the final verdict depends on that premise, and how many tokens the agents spend before it is labeled defeated. The prediction is that thinking-mode agents in our system stop rationalizing it once it shows as defeated, while under FORD their token use rises along with their errors.

The third experiment is knowledge-graph question answering on WebQSP, CWQ, and MetaQA. Agents argue for candidate answers using retrieved triples, and we compare grounded-semantics resolution against maximum-weight clique resolution. The comparison is made on clean questions and on questions with directional contradictions injected, for example an outdated fact superseded by a dated one.

Finally, the noise-sensitivity study plots verdict accuracy against injected edge-error rate. We pair this with human-annotated edge accuracy on 300 sampled debate turns. Together these tell us whether the grounding step, rather than the semantics, is the limiting factor. If verdicts degrade faster than the edge errors that realistic extractors produce, the method needs an uncertainty-aware variant, such as probabilistic labeling over edge confidences, before it is worth deploying.