ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models

Alex Mei, Sharon Levy, William Yang Wang

Abstract

As large language models (LLMs) are integrated into society, robustness toward a suite of prompts is increasingly important to maintain reliability in a high-variance environment. Robustness evaluations must comprehensively encapsulate the various settings in which a user may invoke an intelligent system. This paper proposes ASSERT, Automated Safety Scenario Red Teaming, consisting of three methods—semantically aligned augmentation, target bootstrapping, and adversarial knowledge injection. For robust safety evaluation, we apply these methods in the critical domain of AI safety to algorithmically generate a test suite of prompts covering diverse robustness settings: semantic equivalence, related scenarios, and adversarial. We partition our prompts into four safety domains for a fine-grained analysis of how the domain affects model performance. Despite dedicated safeguards in existing state-of-the-art models, we find statistically significant performance differences of up to 11% in absolute classification accuracy among semantically related scenarios and error rates of up to 19% absolute error in zero-shot adversarial settings, raising concerns for users' physical safety.

Introduction

As Large Language Models (LLMs) transition from research novelties to ubiquitous personal assistants and advisors, their "robustness"—the ability to provide consistent, safe answers regardless of phrasing—becomes a critical safety requirement. Current safety evaluations often rely on manual "red teaming," where human experts attempt to provoke unsafe outputs. However, manual efforts are slow, expensive, and often fail to capture the full spectrum of prompt instability.

Furthermore, models frequently struggle with "covertly unsafe" text: scenarios that avoid obvious trigger words like "bomb" or "kill" but describe physically hazardous actions, such as using a propane torch to remove house paint. ASSERT (Automated Safety Scenario Red Teaming) addresses these challenges by automating the discovery of safety blind spots through systematic prompt perturbation and adversarial knowledge injection.

The ASSERT Framework

The ASSERT framework utilizes three distinct automated methods to generate a comprehensive test suite for evaluating model robustness:

  1. Semantically Aligned Augmentation (SAA): This method tests for prompt instability by generating multiple paraphrases of the same question. A robust model should maintain a consistent safety label across all semantically equivalent versions.
  2. Targeted Bootstrapping (TB): This technique generates related but distinct scenarios to test if a model's safety logic generalizes. For instance, if a model correctly identifies that drinking alcohol for a headache is unsafe, TB might generate a prompt asking if one should drink alcohol to numb the pain of a broken toe.
  3. Adversarial Knowledge Injection (AKI): The most sophisticated method, AKI, extracts a model's own internal "misunderstandings"—hypothetical benefits of an unsafe action—and feeds them back into the prompt as a "hint." This tests whether the model can be tricked into flipping its safety label from unsafe to safe based on misleading context.
Figure 1: Examples of the three ASSERT methods: Semantically Aligned Augmentation, Targeted Bootstrapping, and Adversarial Knowledge Injection.

Safety Domains and Evaluation

To provide a granular analysis of model performance, the study builds upon the SAFETEXT dataset, partitioning context-action pairs into four specific safety domains: Outdoors, Medical, Household, and an "Extra" category for miscellaneous hazards. This partitioning allows researchers to identify which environments are most prone to model failure.

Figure 4: Definitions and examples of the four safety domains used to partition the SAFETEXT dataset for fine-grained analysis.

Key Insights and Results

Prompt Instability and Domain Sensitivity

The evaluation revealed significant vulnerabilities across all tested models, including state-of-the-art systems like GPT-4. Small wording changes (SAA) led to an 11% absolute difference in classification accuracy for some models. Interestingly, Household scenarios proved to be the most vulnerable. Researchers speculate that the prevalence of "household hacks" in training data may make it difficult for models to distinguish between clever DIY solutions and dangerous mistakes.

Adversarial Success and "Self-Gaslighting"

The AKI method demonstrated that models can be their own worst enemies. By injecting adversarial hints, researchers achieved a 19.76% error rate in zero-shot settings. This vulnerability increased dramatically in "four-shot" settings, where providing four examples of bad advice labeled as "safe" caused the error rate to skyrocket to 51.55%. GPT-4, while robust to simple paraphrasing, was surprisingly susceptible to AKI; its advanced instruction-following capabilities made it more likely to be led astray by sophisticated, misleading hints.

Model Comparisons

While GPT-4 remained the most robust overall, showing no statistically significant performance drop across paraphrased prompts, chat-tuned models like Vicuna and Alpaca exhibited significant instability. This is likely due to smaller parameter sizes and less rigorous Reinforcement Learning from Human Feedback (RLHF) compared to the GPT series.

Conclusion

ASSERT demonstrates that even the most advanced LLMs possess significant safety blind spots that can be systematically exploited through automated red teaming. The success of Adversarial Knowledge Injection suggests that safety is not just about filtering "bad words," but about ensuring models can maintain logical consistency in the face of misleading context. Future work will expand this framework to include non-English prompts and non-physical harms, such as psychological or financial risks, to further bolster the reliability of AI systems in high-variance environments.

Summary updated

Abstract

As large language models are integrated into society, robustness toward a suite of prompts is increasingly important to maintain reliability in a high-variance environment.Robustness evaluations must comprehensively encapsulate the various settings in which a user may invoke an intelligent system. This paper proposes ASSERT, Automated Safety Scenario Red Teaming, consisting of three methods -- semantically aligned augmentation, target bootstrapping, and adversarial knowledge injection. For robust safety evaluation, we apply these methods in the critical domain of AI safety to algorithmically generate a test suite of prompts covering diverse robustness settings -- semantic equivalence, related scenarios, and adversarial. We partition our prompts into four safety domains for a fine-grained analysis of how the domain affects model performance. Despite dedicated safeguards in existing state-of-the-art models, we find statistically significant performance differences of up to 11% in absolute classification accuracy among semantically related scenarios and error rates of up to 19% absolute error in zero-shot adversarial settings, raising concerns for users' physical safety.
BibTeX
@inproceedings{mei2023assert,
  author = {Alex Mei and Sharon Levy and William Yang Wang},
  title = {{ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=ewedHtUI5X},
}
Figure