The Devil is in the Context: Nine Dimensions of Prompt Injection Attacks

Jiamin Chang, Kristina Nikolić, Rishi Sharma, Giovanni Cherubin, Vasilios Mavroudis, Jose Sanchez Vicarte, Arnav Garg, Hammond Pearce, Andrew Paverd, Ahmed SalemarXiv 2026 · arXiv

In the rapidly evolving world of AI agents, "prompt injection" has become a catch-all term for a variety of security failures. We often treat it as a single, monolithic threat, but a new paper from researchers at UNSW, ETH Zurich, and Microsoft argues that this oversimplification is dangerous.

In "The Devil is in the Context: Nine Dimensions of Prompt Injection Attacks," the authors demonstrate that an instruction's "maliciousness" isn't just in the text itself—it’s in the context of the user’s intent and the system’s capabilities.

TL;DR

This paper introduces a nine-dimensional taxonomy for indirect prompt injection, shifting the focus from the attack payload alone to the "triplet" of payload, user task, and system context. By applying this taxonomy, the authors reveal that current benchmarks have massive blind spots, missing over 40% of complex attack combinations, and provide a tool to automatically generate more diverse, "real-world" test cases.

What Problem Does This Paper Address?

Current evaluations of prompt injection defenses are often misleading. A defense might boast a 0% attack success rate on a popular benchmark, yet fail miserably against a real-world exploit.

The core issue is that prompt injection is highly contextual. For example, the sentence "Send the completed report to the address below" is a legitimate command if the user asked the agent to do exactly that. However, if the user asked for a summary and the agent finds that sentence hidden in a third-party document and executes it, it’s a successful injection.

By treating all injections as a single type of attack, we ignore critical variations like whether the attack is hidden, whether it modifies the agent's long-term memory, or whether it only triggers under specific conditions. This lack of nuance prevents developers from matching defenses to their specific threat models.

Methods

The researchers propose a taxonomy consisting of nine binary dimensions that characterize an attack based on the instruction, its delivery, its relation to the user task, and its intended effect on the system.

The Nine Dimensions of Prompt Injection

The "unit of classification" here is the most novel aspect: they don't just look at the Injected Instruction; they evaluate it alongside the Authorized User Task and the Agentic System context (trust policies, available tools, etc.).

Contrast between previous taxonomies and the proposed contextual approach

To make this taxonomy practical, they developed:

  1. An Automated Annotation Pipeline: Using a hybrid of deterministic rules and an LLM judge to label attack instances.
  2. Taxonomy-Guided Case Generation: A system that takes a simple "seed" attack and programmatically transforms it into a more complex variant (e.g., making it "Hidden" or "Conditional") to test specific defense gaps.

Datasets

The authors validated their taxonomy across three main areas:

  • Real-World Cases: They analyzed six high-profile documented vulnerabilities, including exploits against GitHub MCP, Claude, and M365 Copilot.
  • Existing Benchmarks: They annotated nine major benchmarks, including AgentDojo, IPIArena, LLMail-Inject, and Trojan Hippo.
  • Generated Variants: They created 40 new targeted variants based on the AgentDojo Workspace suite to expose vulnerabilities that the original, simpler cases missed.

Key Insights and Ideas

  • Benchmark Blind Spots: Most current benchmarks are "easy." They focus heavily on attacks that are visible, single-payload, and unconditional. "Darker" categories—like attacks that are hidden in metadata or those that modify an agent's persistent state—are severely underrepresented.
  • Combinatorial Gaps: Even if a benchmark covers "Hidden" attacks and "Conditional" attacks separately, it rarely tests them together. The authors found that 13.2% of all possible pairwise category combinations are completely missing from the combined benchmark collection.
  • Contextual Ambiguity: The same payload can receive different labels under different user tasks. This proves that "detecting" prompt injection cannot be done by looking at the input string in isolation.
  • Defense Specificity: The taxonomy helps explain why certain defenses fail. For instance, an information-flow defense might stop an "Untrusted source" attack but be completely bypassed by an "Agent modifying" attack that poisons the agent's memory to trigger in a later, "trusted" session.

Key Results

  • Real-World Diversity: The analysis of six public exploits showed that they collectively spanned all nine dimensions, proving the taxonomy's relevance to actual threats.
  • Benchmark Analysis: Figure 2 (below) highlights the uneven coverage. For example, AgentDojo is excellent for testing "Topic Relation," but almost no benchmarks effectively test "Transformation required" (obfuscated) instructions.
Fig. 2. Taxonomy coverage of existing prompt injection benchmarks. Each cell represents the distribution of attack session input across the two categories of one taxonomy dimension. Colors follow Table I respectively, and their relative areas indicating the proportion of instances assigned to each category. Grey indicates cases where the corresponding category cannot be determined at the system level. The four AgentDojo suites are reported separately.
  • Exposing Vulnerabilities: In their generation experiment, the authors took 40 cases where a base model (GPT-5.6-Luna) originally had a 0% attack success rate. By using the taxonomy to generate "On-topic" and "Conditional" variants, the attack success rate jumped to 17.5%. This demonstrates that current benchmarks are failing to exercise the model's true vulnerability surface.

Limitations and Future Work

The authors acknowledge that while nine dimensions are a strong start, they aren't exhaustive.

  • Modality: The current taxonomy focuses on text. Future versions will need to account for multi-modal agents (images, audio).
  • Multi-Agent Systems: As agents begin to talk to each other, "injection propagation" will become a critical new dimension.
  • Automation Hurdles: Some dimensions, like "Source Label" (Trust), are difficult to automate because they depend on internal system policies that aren't always explicit in the dataset.

The paper concludes with a "Call to Action" for the research community: stop reporting aggregate success rates on underspecified benchmarks. Instead, use a shared vocabulary to state exactly which dimensions of the "devilish" context your defense actually covers.

Summary updated

Abstract

Prompt injection is usually discussed and evaluated as a single, homogeneous type of attack. However, vulnerabilities reported in real-world systems show that prompt injections vary widely. Moreover, the same instruction can be a prompt injection in one context and a legitimate request in another, depending on the user's intent and the system's policies. Ignoring this variation leaves gaps in how defenses are evaluated, and can give a false sense of security against real-world attacks. We identify nine dimensions along which prompt injection attacks differ. Together, they form a taxonomy that describes the injected instruction itself, its relation to the user's task, and how it manifests in the target system. Each dimension affects which defenses can detect or prevent an attack. We first use the taxonomy to characterize six real-world prompt injection vulnerabilities, showing how much these attacks differ. We then apply it to nine prompt injection benchmarks and find that most cover only a small region of this space. Combining benchmarks helps, but 13% of pairwise category combinations remain untested, mostly among the less commonly represented categories. Finally, we show that the taxonomy supports automated annotation of benchmarks and targeted generation of new test cases to fill these gaps. The taxonomy is also useful beyond benchmarks. Defense authors can use it to state which types of prompt injection they evaluated against, and system operators can use it to identify which attacks apply to their system and choose defenses to match.
BibTeX
@misc{chang2026devil,
  author = {Jiamin Chang and Kristina Nikolić and Rishi Sharma and Giovanni Cherubin and Vasilios Mavroudis and Jose Sanchez Vicarte and Arnav Garg and Hammond Pearce and Andrew Paverd and Ahmed Salem},
  title = {{The Devil is in the Context: Nine Dimensions of Prompt Injection Attacks}},
  year = {2026},
  eprint = {2611.04672},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04672},
}
Figure