Jailbreak Attack Initializations as Extractors of Compliance Directions

Amit LeVi, Rom Himelstein, Yaniv Nemcovsky, Avi Mendelson, Chaim Baskin

TL;DR

This paper introduces CRI, a novel framework that leverages "compliance directions" within LLMs to create more effective jailbreak attack initializations. By analyzing how attacks converge towards compliance, CRI significantly boosts attack success rates and reduces computational overhead, highlighting the fragility of current LLM safety alignments.

What Problem Does This Paper Address?

Large Language Models (LLMs) are increasingly integrated into various applications, but their deployment has revealed critical security vulnerabilities, particularly their susceptibility to "jailbreak attacks." These attacks aim to bypass safety mechanisms and elicit harmful or unintended outputs from LLMs. While recent research has shown that initializing these attacks with existing successful prompts (self-transfer) significantly improves their effectiveness, the underlying reasons for this improvement have remained unclear. Furthermore, existing methods often rely on arbitrary or hand-picked initializations. This paper addresses this gap by proposing a principled approach to selecting effective initializations by understanding how LLMs respond to compliance and refusal, and how attacks exploit these responses.

Methods

The core of this work is the Compliance-Refusal Initialization (CRI) framework. CRI is built on the observation that safety-aligned LLMs exhibit distinct "compliance" and "refusal" directions in their activation space. Jailbreak attacks, through gradient descent, tend to move prompts along a "compliance direction" to suppress refusal.

CRI's methodology involves:

  1. Understanding Compliance Directions: Analyzing how gradient-based attacks and their self-transfer initializations converge towards similar compliance directions.
  2. Loss-in-the-First-Step (LFS) Metric: Proposing LFS as a measure of how well an initialization projects a prompt towards the compliance subspace. A lower LFS indicates a more effective initialization.
  3. K-CRI Framework: This framework pre-computes a set of K initializations. It involves:
  4. Clustering prompts based on their embeddings.
  5. Training universal jailbreak attacks on these clusters.
  6. Selecting the best-performing initialization from this set for a given prompt based on the LFS metric.
  7. Prompt Clustering: Utilizing constrained k-means clustering on prompt embeddings to group similar prompts, allowing for more targeted initialization.

Datasets

The paper uses the following datasets:

  • HarmBench: Specifically the "standard behaviors" category, used for fine-tuning CRI and evaluating attacks.
  • AdvBench: Used for evaluating attacks and prompt clustering.

Key Insights and Ideas

  • Compliance Directions are Key: Jailbreak attacks don't just randomly find vulnerabilities; they exploit specific "compliance directions" within the LLM's internal representations. Understanding and leveraging these directions is crucial for effective attacks.
  • LFS as a Predictive Metric: The Loss-in-the-First-Step (LFS) metric is a reliable indicator of how many steps an attack will need to succeed. This allows for efficient selection of initializations that are already "closer" to a successful jailbreak.
  • CRI's Efficiency: CRI significantly reduces the computational overhead of jailbreak attacks by selecting highly effective initializations, requiring fewer optimization steps to achieve high success rates.
  • Transferability of Initializations: CRI's pre-computed initialization sets can be transferred across different datasets and models, reducing the need for re-training and highlighting the generalizability of the learned compliance directions.

Key Results

  • Increased Attack Success Rate (ASR): CRI consistently achieves higher ASR compared to standard, random, and baseline initializations across various attacks (GCG, Embedding) and LLMs (Llama-2, Vicuna, Llama-3, etc.). For instance, on Llama-2 with the GCG attack, 25-CRI achieved 100% ASR with an MSS of 2, compared to 78% ASR with an MSS of 122 for random initializations.
  • Reduced Computational Overhead: CRI significantly reduces the number of steps (MSS/ASS) required for successful attacks, leading to substantial GPU hour savings (e.g., approximately 25 GPU hours saved).
  • Robustness to Batch Size: CRI's effectiveness is maintained or improved across different batch sizes, demonstrating its adaptability.
  • Cross-Dataset Transferability: CRI initializations trained on one dataset (AdvBench) perform well when deployed on another (HarmBench) without retraining, showcasing its generalization capabilities.

Limitations and Future Work

  • Diversity of Attacks: The authors acknowledge that initialization-based attack strategies might limit the diversity of produced attacks, potentially leading to narrow attack trajectories. This could inadvertently harm the effectiveness of alignment training against a broader range of threats.
  • Ethical Considerations: The paper highlights the potential for CRI to facilitate harmful data generation and reveal vulnerabilities. They emphasize the importance of this research for promoting AI security and building stronger defenses.
  • Future Work: The authors suggest leveraging CRI to gauge LLM robustness by measuring the minimal computational resources required to achieve effective CRI sets. Further studies could explore how to adapt CRI for alignment training and investigate its impact on model effectiveness against diverse jailbreak threats.

Summary updated

Abstract

Safety-aligned LLMs respond to prompts with either compliance or refusal, each corresponding to distinct directions in the model's activation space. Recent works show that initializing attacks via self-transfer from other prompts significantly enhances their performance. However, the underlying mechanisms of these initializations remain unclear, and attacks utilize arbitrary or hand-picked initializations. This work presents that each gradient-based jailbreak attack and subsequent initialization gradually converge to a single compliance direction that suppresses refusal, thereby enabling an efficient transition from refusal to compliance. Based on this insight, we propose CRI, an initialization framework that aims to project unseen prompts further along compliance directions. We demonstrate our approach on multiple attacks, models, and datasets, achieving an increased attack success rate (ASR) and reduced computational overhead, highlighting the fragility of safety-aligned LLMs. A reference implementation is available at: https://amit1221levi.github.io/CRI-Jailbreak-Init-LLMs-evaluation.
BibTeX
@inproceedings{levi2025jailbreak,
  author = {Amit LeVi and Rom Himelstein and Yaniv Nemcovsky and Avi Mendelson and Chaim Baskin},
  title = {{Jailbreak Attack Initializations as Extractors of Compliance Directions}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure