Introduction
As Large Language Models (LLMs) become increasingly integrated into daily life, ensuring their safety and alignment with human values is paramount. However, current safety benchmarks often fall short due to narrow scopes, a "safety ceiling" where models appear perfectly aligned on simple queries, and evaluation bottlenecks caused by expensive human oversight or inconsistent automated judges.
SALAD-Bench (SAfety evaluation for LLMs, Attack and Defense approaches) addresses these gaps by providing a massive, hierarchical framework designed to stress-test LLMs across 66 distinct harm categories. By introducing "attack-enhanced" queries and a specialized evaluator, the framework reveals significant vulnerabilities in even the most advanced models.
Methodology
SALAD-Bench employs a three-pronged approach to upgrade LLM safety testing:
- Hierarchical Taxonomy: The researchers developed a three-level hierarchy consisting of 6 domains (e.g., Malicious Use, Misinformation), 16 tasks, and 66 specific categories, moving beyond simple keyword-based toxicity detection.
- Question Enhancement: To move past the "safety ceiling," standard harmful queries are augmented:
- Attack Enhancement: Queries are wrapped in "jailbreak" prompts (e.g., "DAN" or gradient-based attacks like GCG) to test the resilience of model guardrails.
- Defense Enhancement: Safety-focused instructions are added to evaluate the efficacy of defensive prompting strategies.
- MD-Judge (Multi-Dimension Judge): A specialized 7B-parameter model fine-tuned on the SALAD taxonomy. MD-Judge acts as an automated safety evaluator, providing precise classifications of why a response is deemed unsafe, often outperforming generalist models like GPT-4 in this specific domain.
Dataset Composition
The benchmark comprises over 30,000 samples:
- Base Set (21k): Standard harmful questions sourced from existing datasets and augmented with 15k self-instructed questions.
- Attack-Enhanced Subset (5k): Questions designed to bypass safety filters using red-teaming methods.
- Defense-Enhanced Subset (200): Targeted samples for evaluating defensive prompting.
- Multiple-Choice (MCQ) Subset (4k): Used to test a model's ability to identify unsafe options in a controlled format.
Key Results and Insights
The "Safety Mirage"
One of the most critical findings is the massive "safety gap." Models that appear 95% safe on standard base questions often see their safety rates drop below 30% when faced with attack-enhanced queries. For instance, Gemini demonstrated high safety on base questions, but its safety rate crashed to approximately 20% under attack, suggesting its guardrails may be more superficial than those of competitors like Claude 2, which maintained an 88.02% safety rate.
Evaluator Performance
MD-Judge achieved an F1 score of 0.818 on the base set, surpassing GPT-4 (0.785) and significantly outperforming LlamaGuard (0.585). This proves that domain-specific fine-tuning is more effective for safety evaluation than raw model scale.
Over-Refusal vs. Utility
The study found that aggressive safety alignment can lead to "over-refusal." Models like Llama-2-13B sometimes refuse to answer even the safe options in multiple-choice tests because they are overly sensitive to the harmful context of the question, highlighting a trade-off between safety and utility.
Conclusion
SALAD-Bench reveals that current LLM safety is often a moving target. While models are becoming better at handling direct harm, they remain vulnerable to sophisticated jailbreak attempts. Future work will focus on expanding the taxonomy to include emerging AI-generated harms and increasing the resilience of automated judges like MD-Judge against adversarial attacks.
Summary updated