When Routine Operations Break Architectural Pattern Relationships in LLM Agent Harnesses

Boming Xia, Qinghua Lu, Dino Sejdinovic, Liming ZhuarXiv 2026 · arXiv

In the rapidly evolving world of LLM agents, we rely on "harnesses"—frameworks like LangChain or DeepSeek—to manage tool calls, state, and safety guardrails. We assume that if we’ve configured a safety plugin, our agent is protected. However, a new paper reveals a startling reality: routine operations like switching a mode or saving a user preference can silently break these architectural guardrails without changing a single line of code.

TL;DR

Routine operations in LLM agent harnesses can cause architectural patterns (like safety guardrails) to fail silently by reordering runtime callbacks or discarding essential state. This study introduces PatternOps, a procedure to identify these vulnerabilities, demonstrating that architectural integrity is a dynamic property of a system's operational history, not just its static configuration.

What Problem Does This Paper Address?

Architectural patterns—such as the Guardrail Enforcement Point (GEP)—are supposed to ensure that certain relationships hold true (e.g., "Policy must check an action before it executes"). In modern LLM harnesses, these patterns are implemented across a mix of native services, third-party plugins, and configuration files.

The authors identify a critical gap: developers often treat these patterns as static. In reality, routine operations—reapplying a plugin, saving "always allow" consent, or compacting conversation history—can shift the "carriers" of these relationships. For instance, a safety plugin might be active, but a runtime operation might move it after the execution point in a callback chain, rendering it useless. The paper calls this "architectural mismatch" at runtime.

Methods

The authors conducted a multiple-case study across five released LLM harnesses (DeepSeek Harness, OpenCode, Claude Code, Pi, and Deep Agents) and 14 independently written plugins.

They introduced PatternOps, a structured assessment procedure that:

  1. Traces Requirements: Maps high-level pattern duties (e.g., from the AgentOps catalogue) to specific provider promises and deployment needs.
  2. Identifies Carriers: Determines what actually "carries" the relationship at runtime—usually either runtime order (the sequence of callbacks) or retained state (saved consent or provenance labels).
  3. Controlled Comparisons: Tests how these carriers behave under operations like mode switches, plugin reloads, and history compaction.

Datasets

The study doesn't use a traditional training dataset. Instead, it evaluates a "main sample" of 14 independently written plugins integrated into 5 released agent harnesses.

  • Harnesses: DeepSeek Harness (rc.2, alpha.2, etc.), OpenCode (2.2.5), Claude Code, Pi, and Deep Agents.
  • Plugins: Includes safety tools like dsh-governance, dsh-airlock, and opencode-permission-reviewer.

Key Insights and Ideas

  • The "Silent Failure" of Mode Switches: In DeepSeek Harness, switching from "observe" to "enforce" mode actually re-registers the policy plugin. This can move the policy behind an approval plugin. If a human operator approves an action, the chain ends, and the policy is never even consulted.
Fig. 1. Policy participation under supported operations. (a) A mode switch reorders callbacks in DeepSeek Harness. Native consent, the host’s own approval question after the plugin chain, keeps policy reachable in enforce mode under the tested read-only profile. (b) A saved allow rule makes OpenCode’s reviewer ineligible. Rows of (b) are matched arms in separate processes.
  • State as a Fragile Carrier: Some guardrails rely on "provenance"—knowing if a piece of data came from an untrusted source. The authors found that operations like history compaction (summarizing old messages to save tokens) can "launder" these labels, making untrusted data appear safe.
  • The "Every-Edit" vs. "Effect Prevention" Distinction: Some architectures only prevent "prohibited effects," but fail the stricter "every-edit" requirement. If one plugin rejects an edit before the policy plugin sees it, the policy plugin loses the chance to "calibrate" or log that attempt, which is a failure in high-assurance environments.
  • History Matters: An assembly's safety isn't just about its current settings; it's about how it got there. Two systems with identical plugins and settings might have different safety profiles because one underwent a plugin reload and the other didn't.

Key Results

The authors quantified these failures across several scenarios (summarized in the paper's tables):

  • Reachability Loss: In DeepSeek Harness, after a mode switch, 0 out of 5 prohibited edits were evaluated by the policy plugin because the callback order had shifted. Disabling and re-enabling the approval plugin "fixed" it by moving it to the end of the chain—a fix that would be undone by the next mode switch.
  • Eligibility Loss: In OpenCode, saving an "always allow" rule for a file caused the policy reviewer to abstain from checking future edits to that file. This meant a prohibited edit could execute without any policy request.
  • Provenance Discard: Reapplying the Airlock plugin (a routine operation during config changes) caused it to discard its private state. Consequently, it failed to restrict edits directed by untrusted content that it had previously flagged.
Table I defines the assessed relationships and their authorities.

Limitations and Future Work

  • Developer Previews: Several harnesses studied were developer previews (e.g., DeepSeek rc.2). While these represent "intended practice," the platforms are still maturing.
  • Manual Effort: PatternOps currently requires significant manual analysis of source code and runtime records to join actions across providers.
  • Future Directions: The authors suggest that harness designers should allow plugins to declare explicit precedence or dependencies (e.g., "Policy must always run before Approval") rather than relying on implicit registration order. They also advocate for "Deny-overrides" logic to be baked into the harness core to prevent any single plugin from accidentally ending a chain before safety checks occur.

Summary updated

Abstract

LLM agent harnesses distribute tool mediation, state management and safety guardrails across native services, independently written plugins and configuration. Routine operations can change the runtime order or retained state that architectural pattern relationships require, without any code change. We study five relationships associated with the Guardrail Enforcement Point, Autonomy Controller and Blackboard patterns across five released harnesses, analysing a main sample of fourteen independently written plugins. Using PatternOps, our assessment procedure, we trace requirements from pattern descriptions, provider documentation and deployment profiles to the providers that realise them and the records that decide them. We then compare alternative allocations under supported operations. Such operations broke relationships silently: after a plugin was reapplied or an allow rule saved, policy stopped evaluating edits while every plugin stayed active, and a prohibited edit executed. Which alternatives qualified depended on the requirement. Allocations that kept the required order, or kept or rebuilt the needed state, held on their tested paths and operation sequences, while others prevented prohibited effects but missed edits rejected elsewhere. In these harnesses, a pattern relationship is a property of the running system and its history, so architects should evaluate it against the decision paths, state lifetimes and operation sequences their deployments use.
BibTeX
@misc{xia2026when,
  author = {Boming Xia and Qinghua Lu and Dino Sejdinovic and Liming Zhu},
  title = {{When Routine Operations Break Architectural Pattern Relationships in LLM Agent Harnesses}},
  year = {2026},
  eprint = {2611.09088},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09088},
}
Figure