TL;DR
This paper argues that "asking nicely" via prompts is insufficient for steering coding agents in industrial or safety-critical environments. By implementing runtime executable contracts—hard gates that block disallowed actions before they execute—researchers achieved zero rule violations across 104 test runs, significantly outperforming prompt-based guidance without sacrificing the agent's ability to solve tasks.
What Problem Does This Paper Address?
As LLM-based coding agents move from suggesting code to actively managing repositories (running shell commands, editing files, and submitting PRs), they gain the power to cause real-world side effects. In regulated industries (e.g., automotive, aerospace, or finance), software development must follow strict engineering and security rules (like "don't use eval()" or "don't modify test files").
Currently, we steer these agents using prose prompts. However, prompts offer two major failures:
- Lack of Guarantee: A model might simply ignore a prompt.
- Lack of Evidence: Prompts provide no verifiable audit trail that rules were actually followed.
The author investigates whether moving from "soft" prompts to "hard" executable contracts can provide the enforcement and evidence required for professional engineering standards.
Methods
The researcher developed a framework using executable contracts: machine-readable rules (defined in YAML) that specify triggers, checks, and failure policies.
The study compared four primary conditions:
- C0 (Control): No rules provided.
- C1 (Prompt): Rules provided as text within the agent's prompt.
- C3 (Post-Generation Check): The agent completes its task, then rules are checked. If they fail, the agent gets feedback and tries to revise.
- C4 (Runtime Enforcement): A "gate" sits between the agent and the environment. Every shell command or file edit is intercepted and checked before it runs. Disallowed actions are blocked immediately.
To evaluate success, the author used an independent oracle. Crucially, the oracle used different detection methods (e.g., Abstract Syntax Tree (AST) scans and import graph analysis) than the in-loop enforcement (which often relied on regex) to ensure the evaluation wasn't just "grading its own homework."
Datasets
To avoid the "data leakage" common in public benchmarks (where LLMs might have seen the answers during training), the author created a custom Python fixture repository.
- Tasks: Freshly written feature requests involving a small web application (handlers, services, and a SQLite store).
- Studies:
- Study 1 (Exploratory): 8 tasks, 240 runs using Gemini 3.1 Flash-Lite.
- Study 2 (Confirmatory): 14 tasks, 112 runs using DeepSeek v4.1 Flash.
- Study 3 (Targeted): 18 tasks specifically designed to trigger known failure modes (like recursive deletes), 108 runs using DeepSeek v4.1 Flash.
Key Insights and Ideas
- Detection is not Prevention: The biggest "aha" moment is the distinction between C3 and C4. If an agent runs
rm -rf /, a post-generation check (C3) can detect the error, but it cannot "un-run" the command. Runtime enforcement (C4) is the only way to handle irreversible actions. - Prompting is Uneven: Prompts worked well for some rules (like "don't touch the test file") but failed miserably for others. For example, when a prompt said "don't use recursive delete" but the system prompt said "you can use
/tmpfor scratch files," the agent often got confused and deleted its scratch directory recursively anyway. - The "Functional Tax" is Low: A common fear is that strict rules will "break" the AI's ability to think or solve problems. The data showed that even when the agent was blocked and had to try a different approach, its success rate (functional pass) did not drop significantly.
- Oracles Need Humans: The study found a "blind spot" where both the enforcement regex and the automated oracle missed a specific type of SQL injection. This highlights that while automated gates are great, they aren't a total replacement for human review.
Key Results
- Zero Violations: In the C4 (Runtime Enforcement) condition, the independent oracle detected zero violations across all confirmatory runs.
- Prompt Failure: In Study 3 (targeted tasks), agents without rules violated policies in 31% of runs. Adding prompts (C1) reduced this, but still left a significant gap that only runtime enforcement closed.
- Irreversible Actions: In Study 1, post-generation checks (C3) reduced file edit violations from 12 runs to 1, but left forbidden shell commands unchanged because they had already executed.
- Cost: The overhead for running these checks was negligible, adding only about $0.004 USD per run in model fees.
Limitations and Future Work
- Model Scale: The study used "Flash" and "Lite" models. While these are realistic for high-volume production, it remains to be seen if larger models (like GPT-4o or Claude 3.5 Sonnet) follow prompts well enough to make runtime enforcement less critical.
- False Positives: The study focused on whether the gate blocked bad actions. Future work needs to measure "false blocks"—how often a gate stops a good action—which could frustrate developers.
- Complexity: The rules tested were relatively simple (regex/AST-based). Enforcing more abstract architectural rules remains a challenge.
- Scaling: The tests were performed on a small fixture. Testing on a massive, multi-million line repository would test the performance latency of the runtime checks.
Summary updated