TL;DR
Reasoning-based LLM guardrails often fail to align their explicit safety rationales with their final moderation decisions—a phenomenon termed the "deliberation-to-enforcement gap." This paper introduces ConsisGuard, a framework that closes this gap by combining high-consistency trajectory distillation with mechanistic functional coupling alignment. The result is a safety guardrail that is not only more accurate at detecting harmful content but also significantly more reliable and internally consistent.
What Problem Does This Paper Address?
Modern LLM guardrails increasingly use reasoning-based approaches (like Chain-of-Thought) to generate explicit safety rationales before outputting a final "safe" or "unsafe" decision. However, the authors uncover a critical safety failure mode: the deliberation-to-enforcement (D2E) gap.
In practice, a model might explicitly detail a severe safety violation in its reasoning but inexplicably output a "safe" label (under-enforcement), or conversely, flag a benign prompt as unsafe without any justification in its rationale (over-enforcement). This differs from general Chain-of-Thought faithfulness because guardrails must strictly execute an external safety policy ($C \rightarrow R \rightarrow Y$, where $C$ is the constitution, $R$ is the reasoning, and $Y$ is the decision). The paper shows that lower policy execution consistency directly correlates with degraded overall guardrail performance and weakened internal representation coupling.
Methods
ConsisGuard addresses the D2E gap at both the data level and the internal representation level via two core components:
- Policy-to-Decision Trajectory Distillation (PTD): The framework uses a teacher model (DeepSeek-R1) to generate synthetic safety trajectories conditioned on a safety constitution, input text, and ground-truth labels. Crucially, these trajectories are filtered using an automated scoring metric ($S_{pec}$) that evaluates both policy grounding (is the reasoning anchored in the policy?) and decision entailment (does the decision logically follow from the reasoning?).
- Functional Coupling Alignment (FCA): To ensure the model's internal mechanisms actually link reasoning to enforcement, the authors use causal tracing to locate the specific attention heads responsible for safety deliberation and decision enforcement. They extract these as "function vectors" and regularize the student model during training to match the representation-level signatures and transition dynamics observed in high-consistency reference trajectories.
Datasets
The authors evaluated ConsisGuard across 10 public benchmarks spanning prompt and response harmfulness detection:
- Prompt Harmfulness Detection: ToxicChat, OpenAIModeration, AegisSafetyTest, HarmBench, and WildGuardTest.
- Response Harmfulness Detection: HarmBench, SafeRLHF, BeaverTails, XSTestResponse, and WildGuardTest.
Training data was synthesized using prompts and responses from the GuardReasonerTrain dataset.
Key Insights and Ideas
- The D2E Gap is a Representation-Level Issue: The authors demonstrate that when a model fails to enforce its own safety reasoning, it isn't just a surface-level text generation error; it corresponds to a measurable weakening or discrepancy in the internal functional coupling between the model's deliberation and enforcement attention heads.
- Mechanistic Alignment Beats Pure Data Scaling: Simply fine-tuning on more consistent data (SFT) helps, but explicitly regularizing the internal causal pathways (via FCA) provides a significant boost in consistency and performance without requiring massive data scaling.
- Localized Regularization Matters: Aligning specific, causally localized function vectors outperforms generic hidden-state regularization (like aligning the entire last layer), proving that safety mechanisms are localized to distinct internal components.
Key Results
- State-of-the-Art Moderation: ConsisGuard achieved strong performance improvements, reaching an average prompt detection F1 score of 81.58% (7B model) compared to the base model's 69.94%, outperforming specialized guardrails like WildGuard (77.68%) and GuardReasoner (80.82%).
- Response Side Success: On response moderation, it achieved an average F1 score of 81.65% (7B model).
- Error Reduction: Ablation studies confirmed that ConsisGuard significantly reduced both under-enforcement and over-enforcement errors, successfully closing the deliberation-to-enforcement gap.
- Cross-Architecture Generalization: The method demonstrated excellent generalization across different model families (Qwen2.5, LLaMA 3.1, LLaMA 3.2) and scales (ranging from 0.5B to 7B parameters).
Limitations and Future Work
- Modality and Scope: The evaluation was strictly limited to text-based, single-turn prompt and response moderation. The authors note that future work should extend this framework to multi-modal, agentic, and multi-turn conversational safety settings.
- Evaluator Bias: The framework relies heavily on LLM-based evaluators to score policy grounding and decision entailment during filtering, which may introduce model-specific biases. Incorporating broader human validation and multi-judge calibration is suggested.
- Computational Cost: Performing causal tracing during training introduces additional analysis and computational overhead.
Summary updated