Backdoors in the Specification Gap

When engineers build a decentralized team of learning agents — a fleet of robots, or a group of large-language-model agents that plan, call tools, and summarize — they usually cannot certify the team as a whole. Instead they certify each agent against its own local specification: a checklist of behaviors that agent must satisfy, small enough to test or even formally verify. The multi-agent safety literature has a sobering result about this practice: local compliance does not add up. Altmann et al. formalize this in Emergence in Multi-Agent Systems: A Safety Perspective, showing that every agent can pass its own check while the joint execution fails the global system specification — agents chase the same reward source or deadlock in a doorway, each one individually blameless. Meanwhile, the backdoor-attack literature has shown that neural models can carry attacker-chosen behavior that stays dormant on normal inputs and activates only under a specific trigger, as in Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks and Trojaning Language Models for Fun and Profit.

These two threads have never been joined, and the junction is dangerous. The specification gap — the space between "every agent is locally compliant" and "the team is globally correct" — is exactly where a backdoor can hide from the auditing infrastructure that real deployments actually have. This proposal claims that an adversary can split a backdoor across a decentralized team: no single agent misbehaves, no single agent fails its local audit, and the malicious behavior exists only as a joint-execution property that activates when trigger components distributed across several agents' inputs co-occur. If true, per-agent certification — the pragmatic default for scaling agent deployments — is structurally insufficient as a security guarantee, and defenses must be redesigned around joint executions.

Core Claim

The claim has an attack half and a defense half. For the attack: there exist poisoning strategies against a multi-agent system such that (i) each agent's policy satisfies its local specification on all inputs, clean or triggered; (ii) clean collective performance is indistinguishable from the uncompromised baseline; and (iii) when all trigger components are present, the joint execution realizes an adversary-chosen global violation. Formally, we extend the emergent-failure condition of Emergence in Multi-Agent Systems: A Safety Perspective, where local specifications $\hat{F}_i$ are each satisfied by policies $\pi_i$ but the global specification $F^*$ fails on the joint policy $\pi = (\pi_1, \dots, \pi_N)$. An emergent backdoor is a trigger $\tau = (\tau_1, \dots, \tau_k)$, with component $\tau_i$ planted in agent $i$'s input stream, such that

$$\hat{F}_1(\pi_1^{\tau}) \wedge \dots \wedge \hat{F}_N(\pi_N^{\tau}) \ \ \text{holds on every input}, \qquad F^*(\pi^{\tau}) \ \ \text{fails exactly when } \tau_1, \dots, \tau_k \text{ all co-occur}.$$

In words: the backdoor is literally an emergent failure in Altmann et al.'s sense, except that the adversary chooses where in the gap it lives and switches it on with a trigger. For the defense: the same offline machinery built to repair benign local-global misalignment can be turned into an auditor of the gap, because benign emergent failures and backdoored ones should leave different fingerprints — benign ones are smooth and symmetric (chasing, deadlocking), backdoored ones are sharp and conditional on a hidden input predicate.

What Is Already at the Intersection

The Lacuna survey of bridging work identifies the near neighbors, and the proposal is positioned against each. Co-Trojan (Gao et al.) fragments a trigger across agents in decentralized policy averaging for deep RL, so the fragments recombine in the weights of a single shared model; that attack lives in parameter space, is detectable by update-anomaly defenses, and never engages the specification structure. Decentralized post-training work (Ersoy et al.) similarly injects at the training-pipeline stage. On the agent side, BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents and Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents attack a single agent's planning, memory, tools, or reasoning loop; there, the compromised agent itself executes the malicious action, so any per-agent behavioral check has a chance to catch it. Finally, the alignment work itself — the formalism of Emergence in Multi-Agent Systems: A Safety Perspective and the coordination machinery of Great Minds Think Alike: Contextual Tacit Communication for Decentralized LLM-Agent Cooperation — assumes all decomposition errors are benign. Nobody has asked what happens when the gap between local and global specifications is occupied on purpose. That is the open problem this proposal takes from the intersection, and the non-adversarial assumption flagged in the alignment literature is precisely the hole we walk through.

Research Program

The first workstream is theoretical: characterize when emergent backdoors exist. The attack is feasible if and only if the specification gap is nonempty on some trigger-conditional region — that is, if there are locally compliant policy perturbations whose composition violates $F^*$. This connects to results that bound the gap: On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning shows local policies approximate global optimality under structured networked interactions, and Hierarchical Mean-Field Deep Reinforcement Learning for Large-Scale Multiagent Systems shows shared population representations shrink destructive interference. These results cut both ways: they constrain our attack surface (tightly coupled, structured teams should be harder to split) and give us a dial — interaction structure — for measuring how attackability scales with the size of the gap.

The second workstream constructs the attacks. In the gridworld settings of Altmann et al. (coin-quadrant and two-rooms), where local specifications and the global specification are explicit, we can search exhaustively for per-agent action modifications that keep every local check green while producing a joint failure on trigger. In LLM agent teams, the trigger components ride the channels that Watch Out for Your Agents! calls observation-attacks, and the poisoning itself uses established non-parametric techniques: corrupting few-shot demonstrations in the prompt, per Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning, or continuous prompt vectors, per BadPrompt: Backdoor Attacks on Continuous Prompts. Stealth of the individual components can borrow from Hidden Backdoors in Human-Centric Language Models, which substitutes visually identical Unicode characters: each component is a commonplace, innocuous event, and only their co-occurrence matters. A key design question is how many agents must be compromised; we expect two of $N$ suffices in most settings, which we will quantify.

The third workstream is a deliberately uncomfortable observation about dual use. The Tact system of Great Minds Think Alike already builds an offline centralized critic that measures the residual between globally optimal joint actions and isolated local preferences, and distills that residual into coordination rules. That residual map is exactly a map of the specification gap — the attacker's search space. The same critic that repairs benign misalignment is the attacker's ideal tool for placing a backdoor where audits do not look, and, symmetrically, the defender's tool for auditing. We will use it in both roles and report the symmetry honestly.

The fourth workstream builds the defense: gap auditing. Offline, before deployment, a centralized critic samples joint states and marks residual-gap regions — places where locally compliant choices compose into global violations. The hypothesis is that backdoored gap regions carry conditional structure (a sharp behavioral switch on a low-dimensional input predicate) while benign emergent failures like chasing and deadlocking are smooth and symmetric, following the failure-mode taxonomy of Emergence in Multi-Agent Systems: A Safety Perspective. A classifier over gap features then flags suspicious regions for targeted red-teaming, converting the needle-in-a-haystack audit problem described in Watch Out for Your Agents! into a much smaller search over flagged regions. We already know one hard limit: if the trigger components are rare homographs as in Hidden Backdoors, offline sampling may never visit the triggered region, and gap auditing fails; we will characterize when this happens rather than hide it, consistent with the decontamination deficits documented in Backdoor Decontamination Dynamics in LLM Agents.

Proposed Experiments

Experiment 1 establishes existence and stealth in the coin-quadrant and two-rooms gridworlds from Emergence in Multi-Agent Systems: A Safety Perspective, where exhaustive search gives ground truth on the gap. Metrics: attack success rate (fraction of triggered episodes where the adversary's global violation occurs), change in clean joint return, and per-agent local-spec compliance under attack — the stealth metric, which we predict stays at 100%. Experiment 2 moves to two- and three-agent LLM teams (a planner, a tool-executor, and a summarizer built in the ReAct style, meaning agents that interleave written reasoning with tool calls) on the agent task settings used by Watch Out for Your Agents! and BackdoorAgent, with the same metrics plus detection rates under each defense. Experiment 3 evaluates defenses at a fixed 5% false-positive rate, reporting detection rate and audit compute cost. Experiment 4 sweeps interaction structure, from loosely coupled teams to the structured networked interactions under which On the Near-Optimality of Local Policies proves local policies near-optimal, testing whether shrinking the benign gap shrinks the attack surface with it.

Baselines

The naive reference point is the undefended team: no monitoring beyond the deployment's built-in per-agent specification checks. This "do nothing" run establishes that the attack works at all — its attack success rate and clean-performance delta are the numbers every defense must be read against, and its 100% per-agent audit pass rate is the headline result.

The strongest attack-side competitor is the single-agent observation-attack of Watch Out for Your Agents!, transplanted into our multi-agent teams: poison one agent's demonstrations with the full trigger and let it execute the malicious action alone. It is the strongest competitor because it is the established attack in the closest setting and it attacks the same observation channel we do. We run it with exactly the same poisoning budget, the same number of corrupted demonstrations, the same trigger token count, and the same compute, differing only in whether the trigger is concentrated in one agent or split across several. Beating it means matching its attack success rate while keeping per-agent local-spec compliance at 100% where the concentrated attack visibly violates the compromised agent's own check. No reported result found on Lacuna for the attack success rate of that paper on its agent benchmarks; the file gives only a generic 90–100% activation range for triggered Trojan behavior, unattributed, so all baseline numbers will be established under matched budgets in this project rather than imported.

On the defense side, the strongest existing competitor is ICLShield from ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks, which targets exactly the poisoning mechanism we use — it shows that models learn demonstrations and adversarial trigger mappings simultaneously — and is the most likely per-agent defense to partially generalize. We also run Fine-Pruning from Fine-Pruning and STRIP from STRIP: A Defence Against Trojan Attacks on Deep Neural Networks per agent, and a black-box global red-team that probes joint trajectories without trigger knowledge. All defenses receive the same audit compute budget and the same false-positive target; beating them means detecting the split trigger at a rate where they sit near chance, and beating the red-team means doing so at lower cost by pointing it at flagged gap regions. Finally, the ablation that isolates our key ingredient holds the poisoning volume fixed and varies only how the trigger is distributed — all components in one agent's stream versus split across two and three — so that any stealth difference is attributable to distribution alone; a second ablation runs the gap auditor with and without the conditional-structure features.

Why not just…?

Why not just backdoor one agent with the existing single-agent attacks? In deployments with no per-agent auditing, yes — the simpler attack of Watch Out for Your Agents! is equally effective and easier, and we say so plainly. Our threat model targets the deployments that do verify agents locally, which is the direction certification of agent components is moving, and where the single-agent attack fails because the compromised agent itself must take an action that violates its own check. The head-to-head transplant experiment settles this empirically: if the concentrated trigger also passes per-agent audits, our attack-side contribution collapses to a measurement study, and we will report it that way.

Why not just monitor the whole team's joint trajectories? A global monitor with trigger knowledge trivially catches any attack; the real question is trigger-agnostic detection, and there the joint trajectory space is exactly the haystack-in-needle problem that Watch Out for Your Agents! documents for reasoning traces, now multiplied across agents. Our gap auditor is a way of telling a joint monitor where to look. This alternative is a genuine threat to the defense half of the proposal: if a black-box red-team with a large but realistic budget achieves the same detection rate as gap auditing, the defense contribution reduces to a characterization of where audits must be spent. Experiment 3 compares both under matched budgets, and we will report the crossover honestly.

Why not just defend at the training level, as in federated-learning backdoor defenses? Those defenses inspect model updates, and the closest existing attack — Co-Trojan in decentralized policy averaging — indeed lives there, which is why update-anomaly methods were built for it. Our attack does not need to touch shared parameters at all: the poisoning channels are demonstrations, retrieved memory, and observation streams, per Universal Vulnerabilities in Large Language Models and the supply-chain steering of Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering, so the updates look clean. The complementary response — close the gap itself with better local specifications — is ruled out at scale by the evidence in Emergence in Multi-Agent Systems: A Safety Perspective, whose symmetry-breaking fixes are demonstrated only in discrete gridworlds, and by the game-theoretic analysis in Revisiting Game-Theoretic Control in Socio-Technical Networks, which shows local utility maximization induces macro-level breakdowns in interconnected networks generally; conjunctive local checks simply cannot express relational global properties.

Decisive Checks

Three outcomes would decide the proposal. If, in the gridworlds where the gap can be enumerated exactly, no locally compliant trigger-conditional joint violation exists, the core claim is false as stated and the attack half ends there. If per-agent monitors in the LLM experiments catch the split trigger at rates well above chance, then the specification gap is not a practical hiding place and the result is a positive one for per-agent certification, published as such. And if benign and backdoored gap regions are statistically indistinguishable to the auditor, the defense half fails and the deliverable becomes a negative result: joint-level auditing is necessary but no cheaper than brute-force red-teaming. Any of these is a finding the field currently does not have.