In the rapidly evolving world of LLM agents, we rely on "harnesses"—frameworks like LangChain or DeepSeek—to manage tool calls, state, and safety guardrails. We assume that if we’ve configured a safety plugin, our agent is protected. However, a new paper reveals a startling reality: routine operations like switching a mode or saving a user preference can silently break these architectural guardrails without changing a single line of code.
TL;DR
Routine operations in LLM agent harnesses can cause architectural patterns (like safety guardrails) to fail silently by reordering runtime callbacks or discarding essential state. This study introduces PatternOps, a procedure to identify these vulnerabilities, demonstrating that architectural integrity is a dynamic property of a system's operational history, not just its static configuration.
What Problem Does This Paper Address?
Architectural patterns—such as the Guardrail Enforcement Point (GEP)—are supposed to ensure that certain relationships hold true (e.g., "Policy must check an action before it executes"). In modern LLM harnesses, these patterns are implemented across a mix of native services, third-party plugins, and configuration files.
The authors identify a critical gap: developers often treat these patterns as static. In reality, routine operations—reapplying a plugin, saving "always allow" consent, or compacting conversation history—can shift the "carriers" of these relationships. For instance, a safety plugin might be active, but a runtime operation might move it after the execution point in a callback chain, rendering it useless. The paper calls this "architectural mismatch" at runtime.
Methods
The authors conducted a multiple-case study across five released LLM harnesses (DeepSeek Harness, OpenCode, Claude Code, Pi, and Deep Agents) and 14 independently written plugins.
They introduced PatternOps, a structured assessment procedure that:
- Traces Requirements: Maps high-level pattern duties (e.g., from the AgentOps catalogue) to specific provider promises and deployment needs.
- Identifies Carriers: Determines what actually "carries" the relationship at runtime—usually either runtime order (the sequence of callbacks) or retained state (saved consent or provenance labels).
- Controlled Comparisons: Tests how these carriers behave under operations like mode switches, plugin reloads, and history compaction.
Datasets
The study doesn't use a traditional training dataset. Instead, it evaluates a "main sample" of 14 independently written plugins integrated into 5 released agent harnesses.
- Harnesses: DeepSeek Harness (rc.2, alpha.2, etc.), OpenCode (2.2.5), Claude Code, Pi, and Deep Agents.
- Plugins: Includes safety tools like
dsh-governance,dsh-airlock, andopencode-permission-reviewer.
Key Insights and Ideas
- The "Silent Failure" of Mode Switches: In DeepSeek Harness, switching from "observe" to "enforce" mode actually re-registers the policy plugin. This can move the policy behind an approval plugin. If a human operator approves an action, the chain ends, and the policy is never even consulted.
- State as a Fragile Carrier: Some guardrails rely on "provenance"—knowing if a piece of data came from an untrusted source. The authors found that operations like history compaction (summarizing old messages to save tokens) can "launder" these labels, making untrusted data appear safe.
- The "Every-Edit" vs. "Effect Prevention" Distinction: Some architectures only prevent "prohibited effects," but fail the stricter "every-edit" requirement. If one plugin rejects an edit before the policy plugin sees it, the policy plugin loses the chance to "calibrate" or log that attempt, which is a failure in high-assurance environments.
- History Matters: An assembly's safety isn't just about its current settings; it's about how it got there. Two systems with identical plugins and settings might have different safety profiles because one underwent a plugin reload and the other didn't.
Key Results
The authors quantified these failures across several scenarios (summarized in the paper's tables):
- Reachability Loss: In DeepSeek Harness, after a mode switch, 0 out of 5 prohibited edits were evaluated by the policy plugin because the callback order had shifted. Disabling and re-enabling the approval plugin "fixed" it by moving it to the end of the chain—a fix that would be undone by the next mode switch.
- Eligibility Loss: In OpenCode, saving an "always allow" rule for a file caused the policy reviewer to abstain from checking future edits to that file. This meant a prohibited edit could execute without any policy request.
- Provenance Discard: Reapplying the
Airlockplugin (a routine operation during config changes) caused it to discard its private state. Consequently, it failed to restrict edits directed by untrusted content that it had previously flagged.
Limitations and Future Work
- Developer Previews: Several harnesses studied were developer previews (e.g., DeepSeek rc.2). While these represent "intended practice," the platforms are still maturing.
- Manual Effort: PatternOps currently requires significant manual analysis of source code and runtime records to join actions across providers.
- Future Directions: The authors suggest that harness designers should allow plugins to declare explicit precedence or dependencies (e.g., "Policy must always run before Approval") rather than relying on implicit registration order. They also advocate for "Deny-overrides" logic to be baked into the harness core to prevent any single plugin from accidentally ending a chain before safety checks occur.
Summary updated