Towards Policy-Compliant Agents: Learning Efficient Guardrails for Policy Violation Detection

Xiaofei Wen, Wenjie Jacky Mo, Yanan Xie, Peng Qi, Muhao ChenSubmission to ICLR 2026 · Rejected

TL;DR

Autonomous web agents often complete tasks while violating critical procedural rules—such as exceeding spending limits or ordering restricted items—that traditional safety filters completely miss. To solve this, the authors introduce PolicyGuardBench, a 60k-example benchmark for evaluating policy violations across web trajectories, and train PolicyGuard-4B, an efficient 4-billion-parameter guardrail that detects violations in just 22.5 milliseconds while matching or outperforming frontier LLMs.

What Problem Does This Paper Address?

Recent work on language agent reliability focuses almost exclusively on catastrophic safety issues: jailbreaking, prompt injections, and generating harmful content. However, real-world autonomous web agents are just as likely to cause damage through policy non-compliance—violating organizational budgets, omitting mandatory user confirmation steps, or failing to adhere to platform constraints.

These violations are difficult to catch because:

  1. They are cumulative and contextual: Adding a single bottle of wine or a second birthday cake to a shopping cart is safe in isolation, but doing so violates an explicit "no alcohol" or "$200 budget" workplace constraint over a full multi-step trajectory.
  2. Existing guardrails fail at this task: Standard safety systems (like Llama Guard) evaluate content toxicity, not logical rule compliance, frequently marking policy-breaching actions as "safe."
  3. Inference latency is critical: Intercepting actions in real time requires fast, lightweight models, whereas general-purpose 70B+ LLMs are too slow and computationally expensive to serve as real-time watchdogs.

Methods

The authors tackle the problem through a systematic data synthesis and modeling pipeline:

  • Trajectory Standardization: Raw browser logs from web agents are converted into a clean, uniform action space (e.g., Click, Input, Submit) with canonical web element names.
  • Policy Synthesis and Matching: Using GPT-4o, the authors generate candidate operational policies across four archetypes: obligations (missing required steps), prohibitions (forbidden actions), ordering constraints, and conditional rules. Policies are then paired with both within-subdomain and cross-subdomain trajectories.
  • Prefix-Based Violation Detection: Trajectories are truncated to early prefixes ($N = 1 \dots 5$ steps) to test whether models can anticipate violations before the entire sequence unfolds.
  • Supervised Fine-Tuning (PolicyGuard-4B): The authors fine-tune a Qwen3-4B-Instruct backbone via full-parameter SFT on PolicyGuardBench, framing compliance checking as a binary classification task (violation vs. no_violation).
  • Efficiency Metric (EA-F1): The authors define Efficiency-Adjusted F1 ($\text{EA-F1} = \frac{\text{F1} \times 1000\text{ ms}}{\text{Latency (ms)}}$) to benchmark the practical trade-off between predictive accuracy and real-time execution cost.

Datasets

The authors construct and release PolicyGuardBench:

  • Source Trajectories: Built from 733 standardized agent trajectories generated by ScribeAgent in WebArena across five platforms: Shopping, Shopping Admin, GitLab, Reddit, and Map.
  • Size & Balance: Curated from over 314,000 raw candidate pairings down to a balanced set of 59,997 trajectory-policy pairs (25,435 violation, 34,562 non-violation; split 80/20 into train and test sets).
  • Cross-Subdomain Coverage: 41.6% (24,985 pairs) specifically test generalization to other subdomains within the same domain.
  • Annotation Protocol: Human-annotated guidelines were used to seed pseudo-labeling with gpt-oss-120B, followed by manual human review for low-confidence and conflicting cases.

Key Insights and Ideas

  • Safety $\neq$ Compliance: Safety guardrails (Llama Guard 3/4, ShieldGemma) completely fail at compliance detection, often dropping below 60% accuracy and labeling nearly all inputs identically. Standard safety checks do not transfer to rule adherence.
  • Specialized Small Models Beat Frontier Giants: At only 4B parameters, PolicyGuard-4B surpasses massive frontier models like DeepSeek-V3.1 (685B), Gemini-1.5-Pro, and Claude-Sonnet-4 in overall compliance accuracy.
  • Early Warning from Action Prefixes: Models can detect impending violations from as few as 1 to 3 actions, though longer prefixes introduce contextual noise that makes classification progressively harder.
  • Zero-Shot Domain Transfer: In a Leave-One-Domain-Out (LODO) setting, the model’s performance only dropped by ~2.5% on completely unseen platforms, demonstrating that it learns general rule-following logic rather than memorizing domain-specific web clicks.

Key Results

  • Benchmark Accuracy & F1: PolicyGuard-4B achieved 90.14% accuracy and an 87.59% F1 score, outperforming Claude-Sonnet-4 (89.83% Acc / 86.78% F1) and Llama-3.3-70B (90.54% Acc / 88.83% F1).
  • Inference Speed: PolicyGuard-4B clocked in at 22.5 ms per sample—more than $13\times$ faster than Llama-3.3-70B (305 ms) and orders of magnitude faster than Claude-Sonnet-4 (1,238 ms) and DeepSeek-V3.1 (3,270 ms).
  • Efficiency-Adjusted F1: PolicyGuard-4B scored 38.93 EA-F1, more than double the closest open-source competitor (Gemma-3-12B at 17.10) and over $50\times$ higher than API frontier models.
  • Out-of-Domain Generalization: In LODO experiments across all five domains, PolicyGuard-4B maintained an average of 90.83% OOD accuracy (compared to 93.28% in-domain).

Limitations and Future Work

  • Detection vs. Intervention: The benchmark and model evaluate passive violation detection. The paper does not explore active intervention, recovery actions, or online steering while the agent is executing.
  • Environment Scope: Trajectories are currently derived exclusively from WebArena environments; real-world commercial web applications with noisy DOMs and anti-bot systems may present messier observations.
  • Proprietary Model Limitations: Certain modern closed-source models (such as Gemini-2.5-Pro) automatically blocked task queries due to aggressive over-defensive safety filters, preventing complete testing.

Summary updated

Abstract

Autonomous web agents need to operate under externally imposed or human-specified policies while generating long-horizon trajectories. However, little work has examined whether these trajectories comply with such policies, or whether policy violations persist across different contexts such as domains (e.g., shopping or coding websites) and subdomains (e.g., product search and order management in shopping). To address this gap, we introduce PolicyGuardBench, a benchmark of about 60k examples for detecting policy violations in agent trajectories. From diverse agent runs, we generate a broad set of policies and create both within subdomain and cross subdomain pairings with violation labels. In addition to full-trajectory evaluation, PolicyGuardBench also includes a prefix-based violation detection task where models must anticipate policy violations from truncated trajectory prefixes rather than complete sequences. Using this dataset, we train PolicyGuard-4B, a lightweight guardrail model that delivers strong detection accuracy across all tasks while keeping inference efficient. Notably, PolicyGuard-4B generalizes across domains and preserves high accuracy on unseen settings. Together, PolicyGuardBench and PolicyGuard-4B provide the first comprehensive framework for studying policy compliance in web agent trajectories, and show that accurate and generalizable guardrails are feasible at small scales.
BibTeX
@misc{wen2025towards,
  author = {Xiaofei Wen and Wenjie Jacky Mo and Yanan Xie and Peng Qi and Muhao Chen},
  title = {{Towards Policy-Compliant Agents: Learning Efficient Guardrails for Policy Violation Detection}},
  year = {2025},
  eprint = {2510.03485},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.03485},
}

Loading related content...

Figure