TL;DR
Even the most advanced "frontier" AI models lose their safety guardrails when they delegate tasks to smaller, cheaper sub-agents. This paper demonstrates that multi-agent systems like Claude Code and Codex are only as safe as their weakest component, as weak sub-agents inadvertently "launder" misinformation, causing strong orchestrators to accept forged data they would have easily rejected on their own.
What Problem Does This Paper Address?
AI safety research has traditionally focused on the vulnerabilities of large language models (LLMs) in isolation. However, the industry is rapidly moving toward multi-agent systems (MAS), where a powerful "orchestrator" model (like GPT-5 or Claude Opus) manages complex tasks by delegating sub-tasks to smaller, more efficient models (like Haiku or GPT-nano).
The authors identify a critical structural vulnerability in this setup: delegation creates a "safety gap." When a strong model reads a document directly, it can spot forgeries or inconsistencies. But when it receives a summarized report from a weaker sub-agent, it often trusts that summary implicitly. This allows an attacker to plant a forged record that the weak agent fails to vet, effectively "laundering" the falsehood into the system's final decision.
Methods
The researchers evaluated this vulnerability using a metric they call harmful revision: a scenario where an agent initially knows the correct answer but changes it to a false one after interacting with another agent or a forged record.
The experimental setup involved a "document store" containing one genuine record and one forged record. The forgeries were designed with "tells" that a sophisticated model should catch—such as missing metadata or dates that predate the original record. They tested two primary conditions:
- Direct: The strong orchestrator searches the store using its own tools.
- Delegated: The orchestrator is restricted from the store and must rely on a report from a weaker sub-agent.
The study used unmodified commercial products (Claude Code and Codex) as well as a custom "reference harness" to ensure the results weren't just quirks of specific software.
Datasets
- Fictional Story Corpus: 180 unique items containing relational facts (e.g., "Who designed the Velra compact?"). Using fictional content ensures the models cannot rely on pre-existing training data.
- Forgery Families: Six types of forged documents, ranging from "glosses" (annotations by a fake archivist) to "stacked" forgeries (multiple documents repeating the same lie).
- Model Panel: 18 models from six developers, including the GPT-5 and Claude 4/5 families, Llama 3, and Gemini 2.5.
Key Insights and Ideas
- The "Laundering" Effect: Weak sub-agents often strip away the "provenance" (the context and metadata) of a document. When they pass a summary to the orchestrator, the orchestrator no longer sees the "tells" that would have signaled a forgery.
- Suppressed Verification: Delegation reduces the orchestrator's "skepticism." In Claude Code, orchestrators took significantly fewer tool-calling turns when delegating, suggesting they "lazily" accept sub-agent reports rather than double-checking the source.
- Channel Semantics Matter: Models are more likely to believe a falsehood if it comes through a "tool call" (a sub-agent's report) than if it comes from a "peer message." We trust the system's internal plumbing more than a direct assertion.
- Negative Correlation of Robustness: Interestingly, a model's ability to resist a forged document does not predict its ability to resist a peer's assertion. In fact, the two traits are negatively related (Spearman $\rho = -0.76$), as seen in Figure 4.
Key Results
The results highlight a dramatic collapse in safety during delegation:
- Direct Access: Strong orchestrators (like Opus 5) were nearly perfect, committing almost zero harmful revisions in over 1,000 trials.
- Delegated Access: When the same orchestrators used weak sub-agents, failure rates spiked. Claude Code failed 75/180 times on "stacked" forgeries, and Codex failed 44/180 on "gloss" forgeries.
- Cascading Failure: In a three-agent chain (Attacker $\rightarrow$ Worker $\rightarrow$ Aggregator), the final aggregator repeated the corrupted worker's error in 98% of trials (751/757).
- The "Grounded" Fix: Forcing the sub-agent to quote the original source record (grounding) or simply instructing the orchestrator to be skeptical dropped the error rate back to near zero.
Limitations and Future Work
- Design Bias: The forgeries were specifically designed to be detectable by "careful readers" (strong models), which may exaggerate the gap between strong and weak models.
- Variance: The authors noted that run-to-run variation can be high, with measurements of the same fact disagreeing 13% of the time.
- Product Constraints: Some products (like Codex) did not allow for mechanical enforcement of delegation, requiring instructional prompts instead.
Future Work: The authors suggest that engineers must avoid placing "cheap readers" between a system and its evidence. Future research should focus on developing "audit-friendly" multi-agent architectures that preserve evidence provenance across every hop in the chain.
Summary updated