Abstract
Large language models (LLMs) are vital for a wide range of applications yet remain susceptible to jailbreak threats, which could lead to the generation of inappropriate responses. Conventional defenses, such as refusal and adversarial training, often fail to cover corner cases or rare domains, leaving LLMs still vulnerable to more sophisticated attacks. We propose a novel defense strategy, Safety Chain-of-Thought (SCoT), which harnesses the enhanced reasoning capabilities of LLMs for proactive assessment of harmful inputs, rather than simply blocking them. SCoT augments any refusal training datasets to critically analyze the intent behind each request before generating answers. By employing proactive reasoning, SCoT enhances the generalization of LLMs across varied harmful queries and scenarios not covered in the safety alignment corpus. Additionally, it generates detailed refusals specifying the rules violated. Comparative evaluations show that SCoT significantly surpasses existing defenses, reducing vulnerability to out-of-distribution issues and adversarial manipulations while maintaining strong general capabilities.
Introduction
Despite rigorous safety alignment, Large Language Models (LLMs) remain notoriously susceptible to jailbreaks—cleverly crafted prompts designed to bypass safety filters. These attacks typically exploit two primary weaknesses: Out-of-Distribution (OOD) Scenarios, such as rare dialects or complex encodings, and Competing Objectives, where models are tricked via role-play or refusal suppression (e.g., "Do not use the word 'sorry'").
Traditional defenses like adversarial training are often reactive, attempting to patch specific vulnerabilities as they are discovered. This research argues for a proactive defense mechanism: a cognitive process that evaluates the underlying intent of a request before deciding whether to fulfill it.
The SCoT Methodology
The proposed Safety Chain-of-Thought (SCoT) transforms the model's response process into a structured three-stage reasoning task:
- Intent Abstraction: The model first summarizes the core intent of the user's request, stripping away distracting "jailbreak" framing or stylistic fluff.
- Safety Regulation Assessment: The model evaluates this abstracted intent against predefined safety categories (e.g., harassment, illicit activities, misinformation) and provides a specific reason for its judgment.
- Final Response: Based on the reasoning, the model either issues a detailed refusal or proceeds with a helpful answer.
To train this capability, the researchers employed Question Evolution. Standard harmful prompts were "evolved" using GPT-4 into complex versions involving slang, uncommon dialects, and persuasive contextual styles like "expert endorsement." Models (Llama-3.1-8B and Mistral-7B) were then fine-tuned on this evolved dataset alongside a "retain dataset" of benign samples to prevent over-defensiveness and maintain general intelligence.
Experimental Evaluation
The models were tested against 26 different attack types sourced from AdvBench and Jailbroken. These included linguistic manipulations (leetspeak, translations) and adaptive adversarial attacks such as GCG (Gradient-based), AutoDAN, and PAIR. To ensure general capability was preserved, the models were also evaluated on benchmarks like MMLU and GSM8K.
Key Results
SCoT demonstrated exceptional robustness across all tested scenarios:
- Near-Zero Attack Success Rate (ASR): SCoT achieved a 0.00% ASR on nearly all linguistic and contextual attacks for the Llama-3.1-8B model.
- Robustness to Adaptive Attacks: Against the formidable GCG attack, SCoT reduced the success rate on Llama-3.1 from 45% (original) to 0%.
- Superiority over Baselines: SCoT significantly outperformed state-of-the-art defenses like Representation Reading (RR) and Adversarial Training (R2D2), particularly in "refusal suppression" scenarios where traditional models are easily tricked into obeying formatting constraints.
- Minimal Performance Loss: On the GSM8K math benchmark, the SCoT-enhanced Llama model scored 85.3%, nearly identical to the original model's 85.7%, indicating that the "alignment tax" was successfully mitigated.
Discussion and Conclusion
SCoT leverages the model's inherent reasoning as a generalizer. By forcing a "thinking" buffer before the final response, SCoT prevents the model from being pushed into a positive start (e.g., "Sure, I can help...") that often leads to successful jailbreaks.
While SCoT introduces a slight increase in inference latency due to the generation of reasoning tokens, it provides a significantly higher safety ceiling. Future work may explore Dynamic SCoT, allowing models to retrieve up-to-date safety policies from external databases to handle emerging threats in real-time.
Summary updated