TL;DR
When multi-agent systems share a memory, the standard "design reflex" is to resolve conflicting facts before they reach the agents. This paper demonstrates that surfacing these contradictions—letting the agents see the disagreement and its provenance—dramatically improves accuracy and robustness against adversarial attacks compared to silently picking a "winner" at the storage layer.
What Problem Does This Paper Address?
In multi-agent debate systems, agents often rely on a shared memory (like a knowledge graph or vector database). A common assumption in the field is that this memory should be consistent. If the system retrieves two conflicting facts (e.g., "The CEO is Alice" vs. "The CEO is Bob"), the storage layer usually employs a heuristic—like choosing the most recent entry or the one with the highest confidence score—to provide a single "clean" answer to the agents.
The authors argue this is a "strange division of labor." A debate system is specifically designed to adjudicate competing claims through reasoning. By resolving conflicts at the storage layer, we use the system's weakest tool (a simple heuristic) to bypass its strongest tool (the reasoning agents). Furthermore, this "silent reconciliation" makes the system brittle; an adversary who knows the tie-breaking rule (like recency) can easily "poison" the memory.
Methods
The authors built DebateGraph, a modular memory service where various mechanisms from current literature can be toggled as independent "switches."
The core experiment compares two policies:
- COLLAPSE (The Baseline): The memory layer ranks conflicting claims and only returns the top-ranked "winner" to the agents.
- SURFACE (The Proposal): The memory layer returns the entire ranked set of conflicting claims, along with their provenance (who wrote it, when, and from what source).
The study used a "gated ablation" protocol: they pre-declared performance thresholds for five different mechanisms (like write-time trust gates and retrieval routers). If a mechanism didn't meet its threshold in controlled runs, it was considered "refuted" or "failed."
Datasets
The researchers evaluated their system using:
- LoCoMo: A dataset of multi-turn conversations.
- HotpotQA: Used for cross-corpus replication to ensure the findings weren't limited to conversational data.
- DebateMem-Conflict: A new, deterministic "poisoning" set introduced by the authors. It contains 122 items where a "lie" is planted that contradicts a gold-standard fact. The lie is attributed to a non-debating writer and given a more recent timestamp to trick recency-based heuristics.
Key Insights and Ideas
- The "Storage Layer" shouldn't be a Judge: Surfacing conflict helps because agents can use the provided provenance to discount unreliable sources or identify contradictions that require deeper reasoning.
- Groundedness is a Trap: One of the "failed" mechanisms was a write-time trust gate designed to filter out "un-grounded" claims. It failed because adversaries are smart: they compose lies using entities already present in the graph. To a simple algorithm, a lie about a known entity looks more "grounded" than a truthful first mention of a new entity.
- The Retrieval Bottleneck: Initially, the benefit of surfacing contradictions seemed small. The authors discovered a "harness bottleneck": their retrieval cap was too low. If the memory doesn't retrieve the contradiction in the first place, the agents can't debate it. Raising the retrieval cap (from 40 to 200 edges) multiplied the positive effect of surfacing by a factor of eight.
- Some "Features" are Bugs: Mechanisms like "router provenance features" actually harmed accuracy once contradictions reached the prompt. Adding complexity to the retrieval logic often introduced more noise than signal.
Key Results
The results were decisive in favor of transparency over silent reconciliation:
- Accuracy Boost: On the graph backend, surfacing contradictions improved the F1 score by +0.340 (a massive jump from the initial +0.043 measured before fixing the retrieval bottleneck).
- Adversarial Robustness: The "SURFACE" approach cut the success rate of adversarial attacks from 22 down to 7 (out of 122 poisoned items).
- Cost Efficiency: These gains were achieved at "equal cost," meaning the token budget for the agents remained the same; the system simply used those tokens to show the conflict rather than hiding it.
As shown in Figure 2, the p3_surface_reach configuration (surfacing with a sufficient retrieval cap) significantly lifted the retrieval ceiling compared to all other configurations.
Limitations and Future Work
- Model Specificity: All tests were performed using the
gpt-oss-120bfamily. While the authors attempted to replicate results with a 70B model, the inference speed was too slow for the full 39-run matrix. - Context Window Limits: In these experiments, the total memory never actually exceeded the model's 32K context window. The authors note that in a regime where memory genuinely overflows, some form of reconciliation might become a physical necessity rather than a design choice.
- Adversarial Sophistication: The planted contradictions were template-generated. Future work could explore how agents handle more "fluent" or subtle adversarial paraphrases.
The paper concludes with a strong recommendation for engineers: Preserve the disagreement; let the reasoner reason. Silently reconciling memory is an assumption worth abandoning.
Summary updated