TL;DR
Conflux is a system-level defense for LLM agents that prevents privilege escalation by tracking the "provenance" (authorship) of all information influencing an action. Unlike probabilistic model-level filters, Conflux uses a deterministic kernel called ITES to ensure an action only proceeds if every principal who contributed information to it is authorized to perform that action.
What Problem Does This Paper Address?
As LLM agents move from simple chatbots to active participants in workflows—reading documents, calling APIs, and managing data—they face a critical security threat: prompt injection. When an agent processes untrusted data (like a malicious email or a compromised website), that data can "hijack" the agent's instructions, leading it to perform unauthorized actions on behalf of the attacker.
The authors argue that current defenses are flawed because they try to solve the problem inside the model. Since LLMs are inherently probabilistic, they can always be tricked by clever, adaptive attacks. Furthermore, the model itself often lacks the system-level context (like who authored a specific document) needed to make a sound security decision. This paper shifts the focus from "detecting malicious intent" to a system-level objective: preventing privilege escalation (PE).
Methods
The core of the Conflux architecture is the ITES (Influence Tracking with Extrapolated Security) kernel. Its operation is based on a few key pillars:
- Principal Contexts: The system tracks the "provenance" of every piece of data the agent consumes. If an agent reads a document by "User A" and a tool output by "Service B," both are added to the current execution's Principal Context.
- The Intersection Rule: Before an action is executed, ITES checks the organization's existing Access Control System (ACS). It admits the action only if every principal in the context is authorized to perform it. If an untrusted "External Principal" influences a sensitive action, and that principal lacks the necessary permissions, the action is blocked.
- Exposure Planning: Since reading untrusted data narrows the agent's allowed actions (monotonicity), Conflux uses planning to order operations. For example, an agent might be directed to perform a sensitive "write" action before reading an untrusted document to ensure the action isn't blocked by the document's author.
- Formal Verification: The authors provide a formal "authority-confinement theorem," proving that if the system correctly tracks influence and mediates all actions, privilege escalation is mathematically impossible for value-based influence.
To evaluate these designs, they introduced two new instruments:
- SLED: A black-box evaluator that treats the LLM as an adversary and exhaustively explores all possible behaviors a defense model might admit.
- SLED-V: A white-box evaluator that checks a serializable intermediate representation (IR) of the system against security properties using solver-backed routes.
Datasets
The authors evaluated Conflux using:
- AgentDojo: A benchmark for realistic agent protocols and prompt injection tasks. The paper notes that their current runs focused on read-only paths to establish a capability floor.
- Custom Finite Models: For the SLED and SLED-V evaluators, the authors generated various environment configurations (varying numbers of principals, inputs, and actions) to test the state-space limits of their defense.
- Reference Benchmarks: The paper discusses related benchmarks like InjecAgent and Tau2-Bench to differentiate their system-level evaluation approach.
Key Insights and Ideas
- System-Level Necessity: The authors prove that no deterministic model-level defense can guarantee security without system-level authorship information. If two identical texts are authored by different people (one trusted, one not), the model alone cannot distinguish them, but the system can.
- Safety vs. Utility: Conflux prioritizes safety. Because it uses a conservative intersection rule, it might block legitimate tasks if the provenance tracking is too broad (a "false denial"). The paper frames this as a trade-off that must be managed through better provenance precision and exposure planning.
- Moving the Trust Boundary: By placing the enforcement point at the "effect boundary" (where the agent actually does something), the system doesn't need to care how a prompt injection happened; it only cares if the resulting action is authorized.
Key Results
- Authority Confinement: The central theoretical result is Theorem 1, which establishes that the ITES kernel prevents privilege escalation under five specific assumptions (Mediation, Completeness, Conformance, Binding, and Known Context).
- Provenance Precision: In symbolic tests (Table 10), the authors showed that moving from a simple "may-have-written" rule to "authenticated provenance" reduced false denials from 35 to 0 in their test cases, while still preventing all 48 vulnerable executions.
- SLED Performance: The SLED evaluator successfully identified "unsafe branches" in deliberately buggy defense models, demonstrating its utility as a security auditor that doesn't rely on the "luck" of a specific LLM's behavior.
Limitations and Future Work
- Control Influence: The current version of Conflux handles "value influence" (data used as arguments) but does not yet fully cover "control influence" (where untrusted data makes the model choose a different tool or logic path entirely).
- Safety-Only: The current guarantee is for safety, not liveness. The system might stop an agent from doing something bad, but it doesn't guarantee the agent will successfully complete its goal.
- Side Channels: The defense does not cover timing attacks or side channels where a principal might infer information based on how long an agent takes to respond or whether an action was denied.
- Future Directions: The authors are working on integrating Conflux with industry-standard policy languages and extending provenance tracking to cover multi-agent collusion and more complex delegation chains.
Summary updated