A tool-using agent can be talked into a harmful job by giving it the job in harmless pieces: query this table, save it as a CSV, upload the file to this address. Each step looks innocent, so per-step refusal disappears. On DecompBench, frontier models refuse about 90% of harmful tasks when asked directly, but only 6% (GPT-5-mini) and 2.5% (Claude Haiku 4.5) once the task is split into sub-actions. Attack success rises from 0% to roughly 70% (Hidden in Plain Sight). A separate line of work shows that aligned models often internally recognize harmful requests even when they comply. This recognition shows up in their hidden activations, the vectors the network computes between input and output, and a cheap linear classifier can read it out (Any-Depth Alignment; Efficient LLM Moderation with Multi-Layer Latent Prototypes). No one has asked whether that internal recognition survives decomposition. Without that answer, we cannot tell whether decomposition attacks succeed because no model ever sees the whole picture, or because a model sees it and still does not act.
This note proposes to measure that directly and then build the defense the measurement points to. The defense is a running "action ledger": a compact log of every sub-action and its effect on the environment. The same base model reads the ledger incrementally, and a linear probe of its hidden state at an injected safety token decides whether to halt. Executors stay narrow and context-isolated, as decomposition requires. Only the ledger holds the global view, and reading it costs one probe on a cached prefix rather than a second guard model or a multi-agent debate.
Core Claim
Decomposition attacks combine two separable failures. The first is an assembly gap: under history reset, no single context contains the harmful whole, so no detector can see it, latent or textual. The second is a selection gap: when the pieces are assembled in one context, the model's hidden state encodes harm but generation does not refuse. We claim that both gaps can be closed by an aggregation step plus a latent readout.
The aggregation step is the ledger. Restoring a global view in a cheap, separate context closes the assembly gap. A safety-token probe on that context closes the selection gap. We further claim that this combination halts decomposed attacks earlier, in fewer executed harmful actions, and at far lower cost than text-level guards or debate.
The second half of the claim is genuinely at risk. A text guard reading the same ledger may match the probe on accuracy. The experiments are designed so that this outcome would be visible rather than hidden.
The idea also addresses a real tension in multi-agent design. Consequence-aware safety requires judging the real-world outcome of an action rather than its surface words (Read the Scene, Not the Script). Multi-agent decomposition, by contrast, deliberately narrows each agent's view to avoid context overload (A survey on LLM-based multi-agent systems). The ledger separates the two: narrow executors for doing, one wide but cheap reader for judging.
Relation to Prior Work
Three works supply the method:
- Any-Depth Alignment (ADA) (Any-Depth Alignment) injects assistant-header "safety tokens" mid-generation and trains a linear probe (ADA-LP) on the resulting states. It halts harmful continuations at about 25 ms of overhead by reusing the key-value cache, the stored attention state of the prefix already processed. We reuse its readout mechanism unchanged. ADA was evaluated only where intent is linguistically complete, in monolithic prompts and prefilled continuations.
- Multi-Layer Latent Prototypes (MLPM) (Efficient LLM Moderation with Multi-Layer Latent Prototypes) shows that which layer carries safety information varies by model family. It peaks mid-network in Mistral and in the final layers in OLMo. We therefore select layers per model rather than fixing one.
- Directed Representation Optimization (On Prompt-Driven Safeguarding) shows that harmful and benign prompts separate in hidden states even without a safety prompt. This underwrites the hope that a probe trained on monolithic data transfers.
Two works motivate the proposal by exposing the gap:
- DecompBench (Hidden in Plain Sight) is a pure attack benchmark. It proposes no defense and never inspects activations. It does report that, under history reset, the environment acts as the memory. This is exactly why our aggregator logs environment effects and not only instructions.
- Jigsaw Puzzles (Jigsaw Puzzles) splits harmful questions into word fragments across turns and reaches 93.76% average attack success. It argues that the model reassembles the request internally, but never probes for this. Jigsaw is our cleanest test of the selection gap, because the full history is present in context.
The closest defenses all operate on text. AutoDefense (AutoDefense) splits response filtering into intention-analysis, prompt-inference, and judge agents. This costs about 7 seconds per check, and it evaluates single responses rather than trajectories. RePD (RePD) decomposes a prompt into a jailbreak template and a core question by matching against 74 known templates, which does not cover multi-step agentic splits. GUARDIAN (GUARDIAN) models multi-agent dialogue as a temporal graph over BERT embeddings of emitted text and flags anomalous agents. It explicitly avoids hidden states and targets hallucination spread, not adversarial fragmentation. Our proposal differs from all three in two ways: it reads the model's own activations, and it is built around the trajectory rather than the turn.
One constraint comes from mechanistic work on latent multi-agent communication (What Do Latent Agents Actually Represent?). That study found that correctness awareness is inherited from the shared base model rather than created by agent interaction, with cross-agent redundancy of $R^2 = 0.75$–$0.80$. We read this as support for running the ledger reader on the same base model. If safety awareness is a base-model property, a separate guard model buys little beyond what the base model already encodes. Another constraint comes from attention-head analysis (Robust Harmful Features Under Jailbreak Attacks): jailbreaks suppress the heads that transmit refusal rather than erasing harm features. This predicts the selection gap and suggests reading features before those heads act.
Research Program
Diagnosis. We run DecompBench trajectories with open-weight executors: Llama-3.1-8B-Instruct, a Qwen3 model including Qwen3-Coder 30B as used in DecompBench, OLMo2-7B, and Mistral-7B. Each trajectory is run under three context conditions. Under full history, the executor sees all prior sub-actions. Under reset, as in DecompBench, it sees only the current subtask. Under ledger, a separate reader context holds a structured log of each action and its environment effect. Examples of ledger entries are "SELECT name, salary FROM employees returned 412 rows" and "POST file to external host."
At every step and in every condition, we append a safety token and record a linear probe score at each layer. The probe is trained only on monolithic harmful and benign prompts from HarmBench (HarmBench) and WildJailbreak, so any detection on decomposed trajectories is genuine transfer. Per condition, we report three quantities: probe area under the ROC curve (AUROC) for harmful versus benign trajectories at each step, the generation-level refusal rate on the same context, and the difference between them. That difference measures the selection gap.
Defense. The ledger monitor runs alongside the executors. It appends each action-and-effect line to its cached context, injects the safety token, and halts the trajectory when the probe score $s_t$ at step $t$ crosses a threshold. The threshold is fixed on benign data to a 2% false-positive budget. Because the reader shares weights with the executor, deployment needs no second model. Per-step cost is one short forward pass over the new ledger line plus a dot product.
Benign twins. A probe that simply fires on "external upload" would look strong on attacks alone. To rule this out, we build a matched benign twin for every DecompBench task. Each twin uses the same services and the same atomic action types, such as exporting payroll to the company's own HR drive, and differs only in outcome. This set is new, and it is the main data contribution.
Baselines
Naive reference. The undefended executor under DecompBench's reset protocol is the starting point. Its expected attack success is roughly the reported ~70%, with refusal of 6% and 2.5% for the closed models (Hidden in Plain Sight). We re-measure all of these on our open-weight executors.
Strongest textual competitor. This is a text guard reading the identical ledger, using WildGuard and Llama Guard 3. MLPM reports average F1 of 90.18 for latent prototypes on OLMo2-7B, against 88.93 for WildGuard and 79.56 for Llama Guard 3, on monolithic jailbreak data (Efficient LLM Moderation with Multi-Layer Latent Prototypes). That margin is small, which is why this baseline is the real threat. It receives exactly the ledger text, its threshold is set to the same 2% benign false-positive budget, and it is called at every step.
Strongest multi-agent defense. AutoDefense is run on the ledger at each step, with LLaMA-2-13b agents as in the original paper. It reduced attack success against GPT-3.5 from 55.74% to 7.95% at a 6.80% false-positive rate on DAN prompts (AutoDefense). No reported result was found on Lacuna for any defense on DecompBench.
Strongest latent method without aggregation. ADA-LP is applied per step inside each reset executor. ADA reports that this reduces attack success from over 50% to below 3% on monolithic attacks, with near-zero false positives (Any-Depth Alignment). This is the ablation that isolates the ledger's contribution.
Probe without the safety token. A probe on the last ledger token's state, with no injected header, isolates the contribution of ADA's readout mechanism.
Safety system prompt. Each executor and the reader receive a safety system prompt instructing them to refuse any subtask that is part of a harmful chain.
Fairness and success criterion. All detectors receive the same trajectories, the same benign-calibrated threshold procedure, and the same tuning data drawn from HarmBench and WildJailbreak. No detector is tuned on DecompBench. We count the proposal as beating a baseline if it achieves lower DecompBench attack success at equal benign-twin false-positive rate, or equal attack success at at least 10x lower added latency. Separately, it must halt attacks before the final exfiltrating action more often than the text guard.
Why not just…?
Why not just run Llama Guard or WildGuard on each agent's input and output? Applied per step, these guards face the same assembly gap as the executor. DecompBench's sub-actions, such as a SQL select or a file upload, violate no policy individually, which is the mechanism behind the collapse to 2.5–6% refusal. The serious version of this alternative runs a guard on the ledger, and that is our main baseline. If it matches the probe on attack success and false positives, we will report that the ledger is the contribution and the latent readout is merely a cost optimization. Even then, re-reading a growing ledger with a separate model costs about 500 ms per 10,000 tokens, versus sub-millisecond to ~25 ms for latent probes (Efficient LLM Moderation with Multi-Layer Latent Prototypes).
Why not just tell every agent to refuse subtasks of harmful chains? Under reset, an agent has no chain to judge, so the instruction has nothing to act on. Safety prompts also shift all inputs along one refusal direction, raising false refusals on benign technical queries to 7.1% in On Prompt-Driven Safeguarding. The benign-twin set will show whether the same over-refusal appears on agentic tasks.
Why not just use a multi-agent judge like AutoDefense over the trajectory? It may well catch these attacks, and it is included as a baseline. However, it costs about 7 seconds per check (AutoDefense), and called per step over trajectories of dozens of actions, that cost is prohibitive. Text-level judges are also open to the persuasion and error amplification that GUARDIAN documents (GUARDIAN). If AutoDefense dominates on accuracy, the latency-versus-accuracy curve will show what that accuracy costs.
Proposed Experiments
The first decisive check separates the two gaps on DecompBench. If probe AUROC is near chance under reset but high under the ledger and full-history conditions, the assembly gap is real and the aggregation step is necessary. If probe AUROC under full history is high while generation-level refusal stays low, the selection gap is real. Jigsaw fragments on the refined Figstep questions (Jigsaw Puzzles) give a second, assembly-free test: the full history is in context, so the latent-versus-refusal difference measures the selection gap alone. Per-layer curves on each model family test whether the mid-network-versus-final-layer pattern reported by MLPM carries over to agentic trajectories.
The second check is the end-to-end defense comparison. Its metrics are:
- DecompBench attack success;
- the median number of harmful actions executed before a halt, which measures the leakage problem ADA raises;
- false positives on the benign twins and on XSTest (XSTest);
- added latency per step.
Benign utility is checked on WebArena-lite, where agentic fine-tuning is known to trade a 20% success gain for a nearly 40% increase in attack success (Unintended Misalignment from Agentic Fine-Tuning). This lets us see whether the monitor falsely halts ordinary web tasks.
The third check uses adaptive attacks against the ledger itself. We extend the DecompBench decomposer in two ways. First, it interleaves benign distractor actions to dilute the ledger. Second, it phrases environment effects euphemistically, the "semantic camouflage" that CB-Bench shows fools keyword-sensitive cues (Read the Scene, Not the Script). We also run a white-box variant that optimizes ledger entries against the probe.
We would consider the proposal falsified under any of three outcomes. The first is that probe-on-ledger performs no better than ADA-LP per step, meaning aggregation adds nothing. The second is that it falls below the text guard on the ledger at equal false-positive rate. The third is that distractor dilution drives its attack success back toward the undefended ~70%.
References
- Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DecompBench
- Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth
- Efficient LLM Moderation with Multi-Layer Latent Prototypes
- Read the Scene, Not the Script: Outcome-Aware Safety for LLMs
- A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges
- On Prompt-Driven Safeguarding for Large Language Models
- Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models
- AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks
- RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process
- GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling
- What Do Latent Agents Actually Represent? Interpreting Hidden-State Communication in Multi-Agent Systems
- Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
- Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigation