Only as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-Agent

Keegan Wang, Anantika MannbyarXiv 2026 · arXiv

TL;DR

Even the most advanced "frontier" AI models lose their safety guardrails when they delegate tasks to smaller, cheaper sub-agents. This paper demonstrates that multi-agent systems like Claude Code and Codex are only as safe as their weakest component, as weak sub-agents inadvertently "launder" misinformation, causing strong orchestrators to accept forged data they would have easily rejected on their own.

What Problem Does This Paper Address?

AI safety research has traditionally focused on the vulnerabilities of large language models (LLMs) in isolation. However, the industry is rapidly moving toward multi-agent systems (MAS), where a powerful "orchestrator" model (like GPT-5 or Claude Opus) manages complex tasks by delegating sub-tasks to smaller, more efficient models (like Haiku or GPT-nano).

The authors identify a critical structural vulnerability in this setup: delegation creates a "safety gap." When a strong model reads a document directly, it can spot forgeries or inconsistencies. But when it receives a summarized report from a weaker sub-agent, it often trusts that summary implicitly. This allows an attacker to plant a forged record that the weak agent fails to vet, effectively "laundering" the falsehood into the system's final decision.

Methods

The researchers evaluated this vulnerability using a metric they call harmful revision: a scenario where an agent initially knows the correct answer but changes it to a false one after interacting with another agent or a forged record.

The experimental setup involved a "document store" containing one genuine record and one forged record. The forgeries were designed with "tells" that a sophisticated model should catch—such as missing metadata or dates that predate the original record. They tested two primary conditions:

  1. Direct: The strong orchestrator searches the store using its own tools.
  2. Delegated: The orchestrator is restricted from the store and must rely on a report from a weaker sub-agent.
Figure 1: The environment. A fictional relational fact attested by one genuine record, with a single forged file placed among 12 genuine ones. In (a) the orchestrator searches the store with its own tools; in (b) it is restricted to its delegation tool, so a weaker sub-agent’s output is the only evidence it receives.

The study used unmodified commercial products (Claude Code and Codex) as well as a custom "reference harness" to ensure the results weren't just quirks of specific software.

Datasets

  • Fictional Story Corpus: 180 unique items containing relational facts (e.g., "Who designed the Velra compact?"). Using fictional content ensures the models cannot rely on pre-existing training data.
  • Forgery Families: Six types of forged documents, ranging from "glosses" (annotations by a fake archivist) to "stacked" forgeries (multiple documents repeating the same lie).
  • Model Panel: 18 models from six developers, including the GPT-5 and Claude 4/5 families, Llama 3, and Gemini 2.5.

Key Insights and Ideas

  • The "Laundering" Effect: Weak sub-agents often strip away the "provenance" (the context and metadata) of a document. When they pass a summary to the orchestrator, the orchestrator no longer sees the "tells" that would have signaled a forgery.
  • Suppressed Verification: Delegation reduces the orchestrator's "skepticism." In Claude Code, orchestrators took significantly fewer tool-calling turns when delegating, suggesting they "lazily" accept sub-agent reports rather than double-checking the source.
  • Channel Semantics Matter: Models are more likely to believe a falsehood if it comes through a "tool call" (a sub-agent's report) than if it comes from a "peer message." We trust the system's internal plumbing more than a direct assertion.
  • Negative Correlation of Robustness: Interestingly, a model's ability to resist a forged document does not predict its ability to resist a peer's assertion. In fact, the two traits are negatively related (Spearman $\rho = -0.76$), as seen in Figure 4.
Correlation between susceptibility to forged documents vs. peer assertions.

Key Results

The results highlight a dramatic collapse in safety during delegation:

  • Direct Access: Strong orchestrators (like Opus 5) were nearly perfect, committing almost zero harmful revisions in over 1,000 trials.
  • Delegated Access: When the same orchestrators used weak sub-agents, failure rates spiked. Claude Code failed 75/180 times on "stacked" forgeries, and Codex failed 44/180 on "gloss" forgeries.
  • Cascading Failure: In a three-agent chain (Attacker $\rightarrow$ Worker $\rightarrow$ Aggregator), the final aggregator repeated the corrupted worker's error in 98% of trials (751/757).
  • The "Grounded" Fix: Forcing the sub-agent to quote the original source record (grounding) or simply instructing the orchestrator to be skeptical dropped the error rate back to near zero.
Comparison of harmful revision rates across different harnesses and conditions.

Limitations and Future Work

  • Design Bias: The forgeries were specifically designed to be detectable by "careful readers" (strong models), which may exaggerate the gap between strong and weak models.
  • Variance: The authors noted that run-to-run variation can be high, with measurements of the same fact disagreeing 13% of the time.
  • Product Constraints: Some products (like Codex) did not allow for mechanical enforcement of delegation, requiring instructional prompts instead.

Future Work: The authors suggest that engineers must avoid placing "cheap readers" between a system and its evidence. Future research should focus on developing "audit-friendly" multi-agent architectures that preserve evidence provenance across every hop in the chain.

Summary updated

Abstract

Major industry multi-agent systems like Codex and Claude Code can be broken by attacks that have long been unsuccessful against modern frontier models. AI Safety as a discipline has largely focused on finding the bounds and vulnerabilities of AI models used in isolation. Yet as humans increasingly deploy these models as agents and multi-agent systems, the same attacks that had been thought to be patched when using a model in isolation can break these same models under specific circumstances. We show that these system breaks are not purely theoretical, but can break Claude Code, Codex, and a reference harness, all unmodified and used with their own sub-agent mechanisms. We show the attack by asking agents to answer questions from a store of files on disk in which we have planted a forged record. Searching the store themselves, the strong model orchestrators commit almost zero harmful revisions in over 1,000 trials. However, if that same orchestrator model delegates to a cheaper sub-agent, which is what these harnesses are built to do, the same models on the same facts fail on 20/180, 44/180, 75/180 and 553/900. Thus, even when the system is driven by a strong model, the system's safety lands near the weak reader's, not the orchestrator's. Simulations over 18 models show the same effect in the delivery channel where five aggregators, who on their own were almost never unsafe, repeat a corrupted sub-agent's error in 751/757 trials. The safety of real-world agentic systems that use strong orchestrator agents and weaker sub-agents, and multi-agent systems in general, is defined not by the safety of the strong agent but by the safety of the weakest agent.
BibTeX
@misc{wang2026only,
  author = {Keegan Wang and Anantika Mannby},
  title = {{Only as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-Agent}},
  year = {2026},
  eprint = {2611.17520},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17520},
}
Figure