TL;DR
Human-in-the-loop (HITL) approval is often treated as a "magic button" for safety, but this paper reveals a startling "intent-to-effect" gap: even when a human clicks "Approve" or "Reject," LLM agents frequently fail to follow through. By auditing the entire execution chain—from the model's logic to the runtime harness and the workflow routing—the authors provide a formal framework and toolkit to ensure that human oversight actually translates into real-world control.
What Problem Does This Paper Address?
In high-stakes environments like defense, aerospace, or finance, we rely on human approval to gate autonomous actions. However, the authors argue that human review is a "procedural formality" unless the system can guarantee two things:
- Refusal Safety: If a human rejects an action, it must stop.
- Authorization Completeness: If a human approves an action, it must actually finish.
Current agentic systems are leaky. A model might forget an approved task during a long conversation, a runtime framework (like CrewAI) might lose a task "branch" when pausing for feedback, or a workflow (like n8n) might have logic bugs that ignore the "Reject" signal entirely. This paper moves beyond "LLM-as-a-judge" evaluations to look at native business-state witnesses—did the database actually update? Did the email actually send?
Methods
The researchers formalize an Audit Contract that treats authorization as an enforceable boundary. They break the agent execution into three layers:
- The Model Layer: Testing if the LLM generates the correct tool calls after receiving a decision.
- The Harness Layer: Testing if agent frameworks (CrewAI, Agno, Mastra, etc.) correctly schedule and resume work after a human interrupt.
- The Workflow Layer: Testing if the hard-coded logic in automation platforms (n8n) correctly routes "Approve" vs "Reject" signals.
The core technique is Paired Decision Auditing: they take the exact same system state and run two parallel universes—one where the human approves and one where they reject—then compare the "native effects" (the actual changes in the environment) to find where the logic diverges.
Datasets
The study utilizes a mix of synthetic benchmarks and real-world logic:
- $\tau$-bench & $\tau^2$-bench: Publicly accessible environments for retail and banking tools.
- n8n Workflows: 100 public workflow sources were replayed to see if their internal logic respected approval nodes.
- Model Panel: They tested three major model families (using names like GPT-6.1-Sol, DeepSeek V4.1 Flash, and Claude Haiku 5.5) across 3,300 fresh approval episodes.
- Harness Screening: 2,352 episodes testing specific agent frameworks' ability to handle "pause and resume" cycles.
Key Insights and Ideas
- The "Illusion of Control": A system can record a "Reject" decision in its logs while the backend workflow continues to execute the protected action. The interface and the execution are often decoupled.
- Completeness is Harder than Refusal: Models are generally good at stopping when told "No" (zero "leaks" in the paired panel), but they struggle to maintain the "identity" of a task when told "Yes," especially if the approval happens several turns after the original request.
- Harnesses are the Weak Link: One of the most surprising findings was that the software surrounding the LLM is often the culprit. For example, CrewAI 1.15.37 omitted 50% of approved effects because its scheduler failed to revisit approved branches after a parallel join.
- Native Witnesses vs. LLM Judges: The authors argue against using an LLM to check if another LLM succeeded. Instead, they use "witnesses"—isolated code snippets that check the actual state of the environment—to provide an objective ground truth.
Key Results
- Model Layer: Out of 300 paired approvals, 7 were left unfinished despite the model having clear authorization.
- Harness Layer: In specific "pause/resume" scenarios, frameworks frequently lost track of work. Google ADK lost an approved branch in 3 out of 3 tasks per model when handling parallel approvals.
- Workflow Layer: 6.8% of policy-comparable n8n workflows (4 out of 59) executed the protected effect even after a "Reject" decision due to faulty routing logic.
- Diagnostic Tool: The authors introduced an "Approval Triage" toolkit that achieved 1.00 accuracy in locating whether a failure happened in the model, the harness, or the workflow.
Limitations and Future Work
- Deployment Prevalence: The paper focuses on "risk-directed" checkpoints rather than estimating how often these failures happen in average daily use.
- Environment Instrumentation: Their method requires "native witnesses," which means the environment must be instrumented to report its state. This is harder to do in "black-box" proprietary SaaS environments.
- Future Directions: The authors suggest investigating "Approval Laundering" (where an agent gets approval for a benign task that transitively triggers a restricted one) and "Loopjacking" (hijacking the HITL process itself).
By providing a formal "audit contract," this research offers a roadmap for engineers to move LLM agents from experimental toys to reliable, auditable production systems.
Summary updated