This paper investigates a sophisticated new frontier in AI security: indirect prompt injection (IPI). While traditional attacks involve placing a full malicious instruction in a document an agent might read, this work asks if an agent can be induced to "self-assemble" a malicious objective from scattered, incomplete fragments.
TL;DR
The authors introduce AdaLCPI, an attack that splits malicious instructions into fragments hidden across long contexts and uses an adaptive evolutionary search to refine them. The attack achieves a 61.4% success rate—nearly double that of explicit-instruction baselines—proving that an agent's ability to synthesize information is a potent, exploitable vulnerability.
What Problem Does This Paper Address?
As LLM agents gain the ability to process massive contexts (100k+ tokens) and use tools like email or web search, they become vulnerable to indirect prompt injection. Most existing research focuses on "explicit" injections, where the full command (e.g., "Delete all my emails") is present in the retrieved text.
However, safety filters and the model's own attention mechanisms often catch or ignore these blatant instructions in long contexts. The authors address a more insidious threat: fragmented injections. If an attacker splits a "bomb" into two harmless-looking components and provides a vague "instruction manual" (reconstruction cue), can the agent's reasoning capabilities be turned against itself to reconstruct the harmful intent?
Methods
The authors propose AdaLCPI (Adaptive Long-Context Prompt Injection), which consists of three main components:
- Fragmentation: A malicious objective $m_g$ is split into two incomplete fragments ($f_1, f_2$) and a reconstruction cue ($c$). For example, $f_1$ might mention a file, $f_2$ might mention a deletion action, and $c$ tells the agent to "combine the fragments marked [X]."
- Long-Context Embedding: These fragments are buried within thousands of tokens of "filler" text retrieved via the agent's tools.
- Adaptive Search Loop: This is the core innovation. Using a framework called OpenEvolve, the attack iteratively refines the fragments and the cue based on actual execution feedback.
- A Judge LLM provides a graded score (1–10) and natural-language feedback on how close the agent got to the goal.
- A Mutator LLM uses this feedback to rewrite the fragments and cue for the next attempt.
Datasets
The attack was evaluated across three tool-using environments: Email, GitHub, and Slack.
- Benchmarks: The study utilized 81 task-objective pairs per model, covering three attack categories: data exfiltration, reputation sabotage, and data removal.
- Models: Seven different models were tested, including GPT-4.1, GPT-5.1, Qwen-3.6-27B, and Gemma-4-31B (note: version numbers reflect the benchmark's specific environment).
Key Insights and Ideas
- The Synergy of Fragmentation and Adaptation: Fragmentation alone is weak, and adaptation on explicit instructions is limited by safety filters. The "magic" happens when they are combined. The fragmented nature helps bypass simple filters, while adaptation ensures the agent actually follows the reconstructed command.
- Agents are "Too Helpful": The very reasoning capabilities that make agents useful—their ability to connect dots across a long document—are what make them susceptible to this attack.
- Stealth through Utility: In many successful attacks, the agent still completes the user's original benign task, making the malicious tool use much harder for a user to notice.
- Distributed Fragments: The attack still works even if fragments are spread across different tool outputs (e.g., one in an email, one in a Slack message), as shown in Figure 5.
Key Results
- Higher Success Rates: AdaLCPI reached a 61.4% macro-average Attack Success Rate (ASR), significantly outperforming the Trojan Hippo-style baseline (32.8%) and AgentVigil (30.0%).
- Model Sensitivity: As shown in Figure 1, AdaLCPI outperformed the baseline on every single model tested, with particularly massive gains on models like Qwen-3.6-27B.
- Filler Length: The attack is most effective at around 8k tokens of filler, suggesting a "sweet spot" where the context is long enough to hide the fragments but not so long that the agent loses the thread entirely.
Limitations and Future Work
- Computational Cost: The adaptive search requires up to 129 target-agent executions per attack, which is resource-intensive for an attacker.
- Defense Evaluation: The paper focuses on the attack's effectiveness; future work is needed to see if specialized "firewalls" or architectural changes (like separating data and instructions more strictly) can mitigate this.
- Scope: While seven models were tested, the behavior of future "frontier" models with even better reasoning might exacerbate this vulnerability.
Summary updated