"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents

Weiheng Wu, Junkai Chen, Zhensu Sun, Jieke Shi, Yunbo Lyu, David LoarXiv 2026 · arXiv

As autonomous coding agents move from experimental toys to production tools, we face a critical security bottleneck: The Approval Problem. If an agent proposes a shell command, how do we know it’s safe?

Current state-of-the-art systems like Claude Code’s "Auto Mode" or guardrail models like ToolSafe rely on analyzing the text of the command. But as a new paper from researchers at Singapore Management University points out, the text is often a lie. A benign-looking make clean might be configured in a malicious Makefile to delete your entire .git directory.

The researchers propose Caper (Consequence-Aware PERmission), a system that stops guessing and starts observing.

TL;DR

Caper is a "consequence-aware" approval mechanism for coding agents that pre-executes shell commands in a snapshotted sandbox to observe their actual effects before committing them. By analyzing file changes and network attempts rather than just command text, Caper reduces harmful outcomes on the SABER benchmark from 14.8% to 6.7% while maintaining a negligible false-blocking rate.

What Problem Does This Paper Address?

The core issue is lack of visibility. Existing automated approvers are "text-based." They look at the command string (e.g., npm install) and the user's request to decide if it's safe. However, the actual behavior of a shell command depends entirely on the state of the workspace and the code it invokes.

Figure 1: The same command can have different effects in different workspaces. The user asks the agent to remove build artifacts and recompile, and in both workspaces the agent proposes make clean && make. The workspaces differ only in their Makefile, whose clean target deletes src/ and .git/ in workspace A but only build/ in workspace B. An approver that sees only the command cannot tell them apart.

As shown in Figure 1, the exact same command text can be harmless in one project and catastrophic in another. If a malicious script is hidden inside a build tool or a third-party package, a text-based approver is effectively blind to the threat. Furthermore, asking humans to approve every command leads to "approval fatigue," where users eventually click "Allow" on everything just to keep working.

Methods

Caper introduces a Transactional Pre-execution workflow. Instead of letting the agent's command run directly on the workspace, Caper intercepts it:

  1. Snapshot: It takes a snapshot of the task's Docker container.
  2. Pre-execute: It runs the command inside the container under strace (system-call tracing).
  3. Bounded Effects Report: It distills the massive raw system-call trace into a concise "Effects Report." This report focuses on "boundary-crossing" consequences: persistent file changes, network connection attempts, and sensitive data reads (like .env files).
  4. LLM Approver: An independent LLM (separate from the coding agent) reviews the report against the user's original task.
  5. Commit or Rollback: If approved, the changes are kept. If blocked, the container is rolled back to the snapshot, and the agent receives a reason for the denial so it can try a safer path.
Figure 2: Overview of Caper. At step t the agent proposes a command at; the task container is snapshotted as st, and at is pre-executed under system-call tracing, with its output held back until Approver decides. Allow commits the executed state and returns the output, and Block restores st and returns the reason so the agent can replan.

Datasets

The authors evaluated Caper using two primary benchmarks:

  • SABER: A benchmark of 682 shell-only tasks where naive fulfillment leads to harms like credential leaks or data destruction.
  • YoloFS-Ext: A new, more difficult benchmark created by the authors. It hides harmful behavior (like deleting source code) inside tools invoked by seemingly benign commands (e.g., hiding a rm -rf inside a compiled binary or a complex Makefile). Crucially, it includes "safe twins"—identical tasks where the tool is actually harmless—to test for over-eager blocking.

Key Insights and Ideas

  • Visibility > Strictness: The failure of current systems isn't that they aren't "strict" enough; it's that they are looking at the wrong data. Even the most advanced LLM cannot predict that a binary named tool will overwrite app.js just by looking at the string ./tool.
  • The "Safe Twin" Test: A key contribution is the realization that a good approver must distinguish between a dangerous command and its "safe twin." Text-based approvers fail this test because the input to the approver is identical for both. Caper succeeds because the observed effects are different.
  • Bounded Evidence: Raw system traces are too noisy for LLMs (often tens of megabytes). Caper's ability to compress these into a ~4KB "consequence card" is what makes LLM-based approval feasible and cost-effective.

Key Results

Caper significantly outperformed all baselines, including rule-based checkers and deployed systems like Claude Code Auto Mode.

  • Lowest Harm Rate: On the SABER benchmark, Caper achieved a 6.7% harmful-outcome rate, compared to 14.8% for Claude Code and over 18% for standard guardrail models.
  • Superior Detection of Hidden Harms: On YoloFS-Ext (where harm is hidden), Caper's harmful rate was 7.8%, while all other systems failed spectacularly, with harm rates ranging from 35.3% to 94.1%.
  • High Precision: Caper blocked only 1.0% of "safe twin" tasks, showing it doesn't just block everything to stay safe—it actually understands the consequences.
  • Low Overhead: The median approval latency is roughly 3.2 seconds, which is comparable to the time an LLM takes to generate a response anyway.

Limitations and Future Work

  • Shell Only: Currently, Caper only gates shell commands. It doesn't yet handle direct file edits (though the authors note these are easier to inspect via diffs).
  • One-Shot Observation: Caper observes one execution. It could be bypassed by "environment-sensitive" malware that detects it's in a sandbox or waits for a specific time to trigger its harmful payload.
  • Irreversible Outbound Effects: While Caper can roll back file changes, it cannot "un-send" a network request that leaves the container during pre-execution. The authors mitigate this by disabling networking in the sandbox, but this limits the agent's ability to perform legitimate network tasks.

Caper represents a shift in how we think about AI safety: moving away from "prompt engineering" safety and toward "systems-level" observability. For anyone building autonomous agents that touch real-world filesystems, this "look before you leap" approach seems like the only viable path forward.

Summary updated

Abstract

Coding agents increasingly rely on automated approvers to decide whether a proposed shell command may run. Existing approvers, from rule-based checkers to guardrail models and deployed permission classifiers, base their decisions on the text of the request and command. Yet a command's behavior depends on the code it invokes and the state of the workspace, and systems that observe file changes leave the assessment to the agent or a human after execution. We present Caper, a consequence-aware approval mechanism that pre-executes each command in a snapshotted sandbox, provides an independent approver with a bounded report of its observed effects, commits allowed commands, and rolls back blocked ones with a reason the agent can act on. Compared with six existing systems, Caper achieves the lowest harmful-outcome rate on SABER (6.7% versus at least 14.8%) and YoloFS-Ext, a benchmark we build in which harmful behavior is hidden inside invoked tools (7.8% versus at least 35.3%), while blocking 1.0% of the safe twins of the dangerous tasks in YoloFS-Ext.
BibTeX
@misc{wu2026shall,
  author = {Weiheng Wu and Junkai Chen and Zhensu Sun and Jieke Shi and Yunbo Lyu and David Lo},
  title = {{"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents}},
  year = {2026},
  eprint = {2610.40753},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.40753},
}
Figure