Quote the Rule First: Guardrails That Resist Distraction From Inactive Policy Clauses

Companies now ask a language model to enforce their own written rules at the moment of use. A small checker model, often called a guardrail, receives a conversation together with a rulebook (no medical dosing advice, refunds only within 30 days, minors may not be shown X) and decides whether the conversation breaks the rules. Real rulebooks are long. For any single conversation, most clauses do not apply, some apply only through an exception, and some come from a different jurisdiction. Separate work shows that language models get much worse when their input contains coherent but irrelevant text. We think the inactive clauses in a rulebook are exactly this kind of text, and that this is a large and unmeasured failure mode of policy-following guardrails. It may also help explain the puzzling finding that larger guardrails do not reliably enforce policies better.

We propose to measure this failure and then fix it. The measurement is how quickly guardrail accuracy falls as inactive, near-miss clauses are added to the rulebook, tracked across model sizes from about 2B to 70B parameters. The fix is to train the guardrail to first quote the specific clauses that govern the case, including any exception that overrides them, and only then give a verdict. Preference training rewards the model for quoting the correct clauses, not just for reaching the correct verdict. In two sentences: inactive rules act as distractors, and we test whether forcing the guardrail to name its governing rule before judging removes that sensitivity at every scale. We also check that it does so better than simply looking up the relevant clause beforehand or using a bigger model.

Core Claim

We make two linked claims. First, guardrail accuracy on a fixed set of conversations falls steadily as semantically close inactive clauses are added to the policy. The rate of that fall does not shrink much with parameter count. Second, a guardrail trained to quote the governing clause set before ruling, using preference pairs built from distracted and clean versions of the same rulebook, will flatten this curve. It will do so more than three alternatives: training on distractor rules without quoting, unconstrained reasoning before the verdict, and selecting clauses with a separate retriever. The advantage should be largest on rulebooks where clauses interact, which is where the retrieval and per-rule alternatives should break.

We summarize robustness with a single number. Let $F_1(k)$ be the guardrail's F1 when $k$ inactive clauses are added to each test policy. The distraction slope is

$$s = \frac{F_1(0) - F_1(K)}{\log_2 (1+K)},$$

that is, the F1 lost per doubling of rulebook clutter up to the largest tested size $K$. We use a log scale because distraction results on retrieval are usually reported over geometrically growing numbers of documents. The proposal succeeds if quoting-first training cuts $s$ substantially at every model size while keeping $F_1(0)$ roughly where it was.

Where This Sits

Three lines of work motivate the gap.

Policy-conditioned guardrails show that following written rules at run time can be learned:

  • STAND-Guard fine-tunes a 7B model on human-readable guidelines.
  • DynaGuard trains 1.7B–8B guardians on 40,000 user-defined policies, with an optional reasoning mode.
  • CoSAlign trains generators to follow safety settings written in the system prompt.

None of these measures how performance degrades as the policy grows with clauses that do not apply. DynaGuard splits policies into separate rules only while building its data. At test time the model reads the whole policy.

The closest prior work is ShieldLM. It adds irrelevant rules during supervised training so that the model learns to pick the rule that applies. We keep that idea as an ablation. ShieldLM differs from our proposal in three ways:

  • It never varies the number or semantic closeness of the distractors.
  • It never asks the model to state which rule governs the case.
  • It never looks at scale beyond 14B.

The distraction literature supplies both the threat model and the method:

  • Breaking Focus uses tree search to generate coherent irrelevant text, which causes roughly a 45% average accuracy drop on frontier models. It also reports that prompts telling the model to ignore irrelevant information do not help, while preference training on distracted versus clean pairs does. That constrains our design and supplies our training signal.
  • Ext2Gen shows that training a retrieval-augmented model to extract evidence before answering beats separate compression modules under heavy noise. We transfer that extract-then-act idea from retrieved passages to rulebook clauses. The change matters because policy clauses can override each other, which retrieved facts rarely do.
  • Llama See, Llama Do traces distraction in Llama-3-8B to a small set of attention heads that boost tokens copied from the prompt. Removing those heads is a training-free competitor we must beat. The same paper's report that backup circuits sometimes take over after removal is a reason to expect that removing heads will not carry over cleanly to larger models.

The scaling literature supplies the evaluation lens:

  • GuardSet-X finds Granite Guardian 3B outperforming 5B on policy-grounded tasks, but never asks why.
  • Acceptance Cards shows that safety fine-tuning claims on a 2B model routinely fail checks on fresh topics and on transfer to new tasks.

We adopt its clustered bootstrap and fresh-domain checks so our claims do not join that list.

Research Program

The benchmark. The first piece of work is a stress-test benchmark called Cluttered Policies, built from existing labeled policy data. We start from the DynaGuard DynaBench test set and from policies in the POLY-GUARD benchmark of GuardSet-X. To each policy we append $k \in \{0, 4, 16, 64\}$ inactive clauses, taken from three sources:

  • clauses drawn at random from other policies;
  • clauses chosen to be close in meaning to the active rule;
  • clauses produced adversarially with the tree-search procedure of Breaking Focus, adapted so that each candidate clause must remain non-applicable.

Gold labels are kept only when a separate check confirms that the transcript does not violate any added clause. That check is a strong model acting as judge plus human audit of a stratified sample. Disagreements are discarded.

The benchmark has a second slice for interacting clauses. Here we insert exceptions, overrides, and scoping clauses (for example, "the rule above does not apply to verified clinicians"). These pairs are minimally edited so that adding one clause flips the gold label. The construction follows the policy-flip pairing of LlavaGuard and the minimal-edit pairs of R2-Guard's TwinSafety. On this slice, judging any single clause in isolation gives the wrong answer by construction.

The method. Quote-then-judge training runs in two stages:

  1. Supervised fine-tuning teaches the output format: the identifiers and text of the governing clauses, then PASS or FAIL.
  2. Preference optimization in the style of DPO follows. Preferred responses quote the correct clause set and give the correct verdict. Rejected responses either quote an inactive or overridden clause, or reach the right verdict through the wrong clause.

Penalizing the right verdict reached through the wrong clause is the key ingredient. It stops the model from learning shortcuts tied to the wording of the transcript.

We train on Qwen3 at 1.7B, 8B, 14B and 32B, matching DynaGuard's model family. We also train Llama-3.1 at 8B and 70B, using LoRA adapters at the largest size. All runs use one fixed data budget.

Analysis across scale. For each model, we record how much attention falls on the governing clauses compared with inactive clauses, and whether that share predicts errors at each size. This tests one candidate explanation for the stagnation reported by GuardSet-X: larger models spread attention over long policy text. The alternative explanation is that larger models have stronger built-in safety habits that override the written rule. We separate the two by comparing errors on permissive policies, which require the model to set aside its default caution, against errors on restrictive policies.

Baselines

Naive reference. Each base model is prompted with the full cluttered rulebook and an explicit instruction to judge only against applicable clauses. It uses the same input formatting as every other condition. This is the condition that Breaking Focus and the in-context alignment baseline of CoSAlign suggest will be weak: CoSA-Score 0.217 versus 0.408 for the trained model on Llama 3.1 8B. We still measure it at every scale, because the "bigger is enough" hypothesis lives here.

Strongest existing method. The strongest competitor is DynaGuard in both its fast and reasoning modes. It is the strongest published model for following user-defined policies, and the file reports DynaGuard-8B at 73.1% F1 on DynaBench versus 13.1% for LlamaGuard3. We use the released checkpoints where available and also retrain the recipe at 14B and 32B on our data budget. DynaGuard has no published results on cluttered rulebooks, so its distraction slope is new. Quote-then-judge must match DynaGuard's $F_1(0)$ within confidence intervals and cut its slope by a margin outside the policy-clustered bootstrap interval.

ShieldLM-style distractor training. This baseline uses supervised training with irrelevant rules mixed in, as in ShieldLM. It is applied to the same base models with the same examples and tuning budget. It doubles as the ablation that isolates our key ingredient, because it uses identical cluttered data with no quoting step and no clause-level preference signal. ShieldLM reports 89.3% overall accuracy on its own test set, against 79.2% for GPT-4 and 50% for GPT-4 on strict rules. We will also evaluate on that test set for continuity.

Second ablation. A model trained with quoting supervision but rewarded only on the verdict tells us whether the clause-level reward matters beyond the output format.

Training-free mechanistic baseline. We remove the entrainment heads identified by the method of Llama See, Llama Do on Llama-3.1-8B. We then repeat the localization at 70B. Head removal must not cost more than one point of MMLU to count as a fair competitor; the original paper reports 56.3% to 55.5% at 8B.

Fairness controls. Every trained baseline sees the same base models, the same pool of training policies, and the same number of gradient steps. Learning rates are searched over the same grid.

Why not just…?

Why not just retrieve the relevant clause before calling the guardrail? This is the most serious threat. On the first slice, where the added clauses are truly unrelated, a good retriever followed by a short-context guardrail may match our method at lower cost. We will report that honestly. The retrieval literature gives grounds to expect a ceiling: Ext2Gen found separate compressors (Recomp, CompAct, EXIT) losing to joint extraction under heavy noise, improving Llama-3.1-8B from 0.473 to 0.557. In policies the structural problem is worse, because an exception clause rarely looks similar to the transcript it governs. The interacting-clause slice settles the question. We run dense and keyword retrievers at depths 1, 3 and 5 in front of DynaGuard. If retrieval matches quote-then-judge on that slice too, we will reframe the method as a cheaper retrieval recipe rather than claim a training advance.

Why not just evaluate each rule separately in parallel, like superposition prompting? Superposition Prompting processes documents on independent paths and prunes the irrelevant ones, and it removes distraction by construction. The same paper reports trouble on multi-hop questions that need interaction across documents. Overrides are exactly such interactions: a FAIL from one rule can be cancelled by a later exception clause. We implement per-clause verdicts with logical OR aggregation as a baseline and expect it to win on the first slice and lose clearly on the second. Its cost also grows linearly with the number of clauses, which we report alongside accuracy.

Why not just use a larger model? The distraction slope at each size answers this directly. Prior evidence points against it:

  • Breaking Focus drops GPT-4o-mini on CommonsenseQA from 85.7% to 22.0%.
  • AIR-BENCH 2024 finds no model fully aligned with the EU AI Act categories.
  • GuardSet-X finds a 3B guardrail outperforming its 5B sibling.

If the slope does fall close to zero by 32B–70B, that is itself a publishable answer to the stagnation question. It would mean the proposal should target efficient small guardrails rather than general robustness.

Why not just tell the model to ignore irrelevant rules, or let it reason freely? The first option was tested and found ineffective by Breaking Focus, and it is our naive reference. Free reasoning is DynaGuard's reasoning mode, which we run directly. The ablation that keeps quoting but rewards only the verdict separates "any reasoning helps" from "being graded on naming the right rule helps."

Decisive Experiments

Experiment 1: distraction curves across scale. We plot $F_1(k)$ for every model and method on both slices of Cluttered Policies, with 95% intervals from a bootstrap that resamples whole policies, following Acceptance Cards. There is no reported result on Lacuna for any guardrail on a distracted DynaBench or POLY-GUARD, and GuardSet-X reports its F1 values only in a figure. All slope numbers are therefore new. The headline claim fails if either of the following holds:

  • the ShieldLM-style ablation reaches a slope within interval of quote-then-judge;
  • quote-then-judge loses more than two F1 points at $k = 0$ against DynaGuard-8B's 73.1% on the clean DynaBench test set.

Experiment 2: fresh domains and transfer. We train on policies from six POLY-GUARD industry domains and test on the remaining two. This mirrors the fresh-subject check in Acceptance Cards. We also test transfer to generators: a CoSAlign-style generator on CoSApien is given cluttered system configurations, with and without a quote-then-judge guardrail giving feedback. This follows DynaGuard's recovery loop, which raised IFEval accuracy from 57.3% to 63.8%.

Experiment 3: adversarial pressure. We rerun the PAIR and AutoDAN attacks used in GuardSet-X against each guardrail, with rulebooks of 16 and 64 clauses. GuardSet-X reported attack success above 90% for Granite Guardian 5B and 19.8% for WildGuard. The attack tests whether naming the governing clause also makes it harder for an attacker to point the guardrail at a clause that does not apply.

Experiment 4: mechanism. At each scale, we test whether the share of attention on governing clauses predicts per-example errors. We also test whether quote-then-judge raises that share. A positive answer at 8B but not at 70B would mean the mechanism changes with scale. In that case we would not claim a single explanation for the stagnation finding.