Guarding the Meaning: Self-Supervised Training for Semantic Robustness in Guard Models

Cristina Pinneri, Christos Louizos

Introduction

As Large Language Models (LLMs) are increasingly integrated into production environments, "guard models" have become critical infrastructure. These models act as secondary safety filters, scanning LLM outputs for toxicity, bias, or harmful content. However, recent research reveals a significant vulnerability: semantic fragility. Even state-of-the-art guard models often flip their safety verdicts based on minor, semantically irrelevant wording changes. For instance, a model might correctly identify a refusal as safe, yet label a semantically identical paraphrase as unsafe. This paper introduces a self-supervised training framework designed to mitigate this fragility by teaching models to reason about intent rather than surface-level linguistic form.

The Proposed Framework

The researchers propose a multi-stage self-supervised pipeline for semantic robustness training. This approach is particularly valuable because it does not require expensive, human-labeled datasets to improve model consistency.

Overview of the self-supervised semantic robustness training pipeline.
  1. Paraphrase Generation & Filtering: The process begins by using an LLM to generate diverse stylistic and syntactic variations of a given response. To maintain data integrity, a high-precision "LLM judge" filters these candidates, ensuring only true semantic paraphrases are retained.
  2. Consistency Regularization: The framework forces the guard model to produce consistent safety scores across an entire paraphrase set, penalizing variance in its predictions.
  3. Parameter-Efficient Tuning: To keep computational costs low, the framework utilizes Low-Rank Adaptation (LoRA), allowing for effective fine-tuning without updating the entire model's parameters.

Skew-Aware Conservative Aggregation

A core challenge in consistency training is determining the "target" score for a set of paraphrases. Standard statistical measures like the mean or median can be problematic in safety-critical contexts. If a model is inconsistent, simple averaging might push safety scores upward, creating a dangerous "optimistic bias" that labels unsafe content as safe.

Comparison of skew-aware aggregation against standard mean and median targets across different distribution types.

To solve this, the authors introduced Skew-Aware Conservative Aggregation. By analyzing the distribution (skewness) of scores within a paraphrase set, the training signal is biased toward the more conservative (less safe) cluster. For example, in a right-skewed distribution where most paraphrases are rated as unsafe but a few are rated as safe, the skew-aware target remains low, ensuring the model does not learn to ignore potential risks.

Experimental Setup and Datasets

The framework was evaluated using several key benchmarks:

  • ToxiGen: Used to generate the initial responses for the paraphrase training sets.
  • BeaverTails (30k_test): The primary benchmark for evaluating safety accuracy.
  • STS-B: Used to validate the reliability of the LLM-based paraphrase filtering.
  • Out-of-Distribution (OOD) Styles: To test generalization, the authors evaluated the model on unseen styles, including Shakespearean English, Legalese, and "Pirate Talk."

Key Results and Insights

The proposed method yielded significant improvements across multiple dimensions of model performance:

  • Reduced Semantic Variability: The framework reduced score variance across paraphrases by approximately 58%, indicating a much more stable internal representation of safety.
  • Improved Accuracy and Calibration: Unlike many robustness techniques that sacrifice performance for stability, this method improved benchmark accuracy on BeaverTails by an average of 2.5%. Furthermore, it reduced Expected Calibration Error (ECE) by up to 40%, meaning the model's confidence scores became much more representative of its actual accuracy.
  • Label Flip Rate (LFR) Reduction: The frequency with which a paraphrase changed the model's final safety verdict dropped dramatically. For the Granite Guardian 8B model, the LFR fell from 27.87% to 9.14%.
  • Generalization: The model demonstrated an ability to generalize its robustness to wildly different linguistic styles (e.g., "Pirate Talk") that were never encountered during the fine-tuning phase.

Conclusion and Future Work

This research demonstrates that semantic robustness is not just a secondary feature but a fundamental component of model safety and calibration. By implementing skew-aware consistency training, guard models can become significantly more reliable against natural linguistic variation. Future work will look to integrate these techniques with adversarial training to defend against intentional "jailbreaking" attempts, further hardening LLM safety infrastructure.

Summary updated

Abstract

Guard models are critical for ensuring the safety of large language model (LLM) outputs, yet they remain vulnerable to superficial linguistic variation. We show that semantically equivalent paraphrases can cause large fluctuations in guard model safety scores, revealing a lack of semantic grounding. To address this, we introduce a two-stage framework: (1) a paraphrasing-based evaluation protocol that quantifies semantic robustness, and (2) a robust training strategy that enforces paraphrase consistency through self-supervised regularization. Our method constructs paraphrase sets for each response, computes a conservative set-level target probability via a skew‑aware estimate, and applies parameter-efficient fine-tuning to align guard model predictions across these variants. Our approach reduces rewording‑induced variability and even improves benchmark accuracy, whereas naive targets such as the mean or median can degrade accuracy in our ablations. These results motivate treating semantic robustness as a first‑class objective and offer a practical, parameter‑efficient recipe for guard models that prioritize meaning over surface form.
BibTeX
@misc{pinneri2025guarding,
  author = {Cristina Pinneri and Christos Louizos},
  title = {{Guarding the Meaning: Self-Supervised Training for Semantic Robustness in Guard Models}},
  year = {2025},
  eprint = {2511.10665},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.10665},
}
Figure