Automated Discovery of Model Specific Safety Vulnerabilities

This direction is split into other directions: non-uniform safety alignment. Its last version stays readable and citable here.

Version 1 of 1113 papers139 conceptsfrozen 2026-10-04

In the early days of Large Language Model (LLM) safety, alignment was often treated as a binary state: a model was either "safe" or "unsafe." However, as models have scaled, researchers have discovered that safety is not a uniform shield but a patchy, inconsistent landscape. A model might be perfectly resilient against hate speech while remaining trivial to "jailbreak" for intellectual property theft or malware generation.

This research direction focuses on the heterogeneity of safety alignment—the fact that vulnerabilities are often specific to a model's architecture, training data, and the linguistic context of the prompt. To map this landscape, researchers are moving away from static benchmarks toward automated, dynamic red-teaming frameworks that use AI to find the "blind spots" in other AI.

The Heterogeneity of Safety Alignment

The core insight driving this field is that safety performance varies drastically across different categories of harm. As Ahmed Magooda and colleagues demonstrate in A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications, a model's ability to prevent hate speech does not predict its vulnerability to data leakage. This "lopsided security posture" means that general safety training often leaves specific domains under-protected.

This inconsistency extends across languages and modalities. Felix Friedrich and Patrick Schramowski found in LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Gaps that models safe in English often harbor significant vulnerabilities in other languages. Similarly, in the world of image generation, Manuel Brack showed that filters are much more effective at blocking sexual content than political misinformation or harassment, as detailed in Distilling Adversarial Prompts from Safety Benchmarks.

From Static Benchmarks to Dynamic Agents

Because vulnerabilities are model-specific, static "multiple-choice" safety tests are increasingly obsolete. Attackers don't use static prompts; they adapt. Consequently, the research has shifted toward automated red-teaming, where one LLM acts as an "attacker" to find the specific linguistic "keys" that unlock another model's weights.

Several frameworks have emerged to automate this process:

  • Multi-Agent Competition: Peiyan Zhang introduced GuardVal, which uses an Adam-inspired optimizer to iteratively refine attacks, preventing the "stagnation" that occurs when an attacker keeps trying the same failed strategies.
  • Reverse Generation: Zhexin Zhang and others proposed flipping the generation process in Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation, creating contexts that are specifically designed to induce safety failures.
  • Scenario Shifting: Attacks like GeneShift by Tianyi Wu use genetic algorithms to evolve "contextual disguises"—wrapping a harmful request in a benign-looking scenario (like a historical reenactment) that the target model is uniquely susceptible to.

Why Jailbreaks Work: The Mechanism of Failure

Why does a model that "knows" it shouldn't help make a bomb suddenly comply when asked to do so in the style of a Shakespearean play?

Nathalie Maria Kirch and Stephen Casper argue in What Features in Prompts Jailbreak LLMs? that jailbreaks exploit complex, non-linear features in a model's internal representations. They found that different attack styles (like role-playing vs. character encoding) exploit distinct, non-overlapping internal pathways. This explains why a defense against one type of attack rarely generalizes to others.

Furthermore, the act of updating a model can inadvertently shatter its safety. Rima Hazra demonstrated in Sowing the Wind, Reaping the Whirlwind that "knowledge editing"—updating a model with new facts—can act as a surgical jailbreak, accidentally disabling the refusal mechanisms for related harmful content.

The Cost of Over-Alignment

As developers push for "safer" models, they often run into the problem of over-refusal. This occurs when a model becomes so sensitive to potential harm that it rejects benign prompts (e.g., refusing to explain "how to kill a process in Linux" because it contains the word "kill").

Justin Cui and colleagues quantified this in OR-Bench, showing a clear trade-off: the models that are most "safe" against toxic content are often the least helpful for legitimate tasks. This has led to the concept of the "Refusal Gap," explored by Xiaohu Du in Refusal-Aware Red Teaming, which measures the discrepancy between a model's internal recognition of harm and its external decision to refuse.

The goal of this research area is to move toward "scenario-adaptive" safety—where models don't just follow rigid rules, but understand the nuance of the context, allowing them to remain helpful while closing the specific, heterogeneous gaps that automated attackers are so adept at finding.

Papers in this version

Showing 60 of 113 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_automated_discovery_of_model_specific_sa_0d82c9fe,
  title = {Automated Discovery of Model Specific Safety Vulnerabilities},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-discovery-of-model-specific-safety-vulnerabilities/rdr_0d82c9fe90a14f3686d797716e0a7d0a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_automated_discovery_of_model_specific_sa_0d82c9fe_v1,
  title = {Automated Discovery of Model Specific Safety Vulnerabilities},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-discovery-of-model-specific-safety-vulnerabilities/rdr_0d82c9fe90a14f3686d797716e0a7d0a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}