Improved Generation of Adversarial Examples Against Safety-aligned LLMs

Qizhang Li, Yiwen Guo, Wangmeng Zuo, Hao Chen

Introduction

Safety-aligned Large Language Models (LLMs), such as Llama-2-Chat, are meticulously trained to refuse harmful requests. However, they remain vulnerable to "adversarial prompts"—carefully crafted suffixes that trick the model into bypassing its safety filters. The current gold standard for generating these prompts is Greedy Coordinate Gradient (GCG), a white-box attack that uses gradients to identify the best tokens to swap.

A significant challenge in this process is the "gradient-token gap." Because text is discrete, gradients are calculated as if the input were continuous, leading to discrepancies between predicted loss reductions and actual outcomes when tokens are swapped. This often results in suboptimal token selection, limiting attack effectiveness against robust models.

Methodology: Bridging the Gradient-Token Gap

Researchers have introduced a new perspective: the gap between gradients and token swaps in LLMs is mathematically analogous to the gap between substitute and victim models in black-box image attacks. By adapting techniques from computer vision, they propose two primary methods to refine adversarial generation.

Language Skip Gradient Method (LSGM)

In modern transformer architectures, each layer consists of a residual module (Attention or MLP) and a skip connection. The authors discovered that gradients from these two paths often conflict, exhibiting negative cosine similarity. LSGM addresses this by applying a decay factor to the gradients flowing through the residual modules during backpropagation, effectively prioritizing the signal from the skip connections.

The transformer layer architecture highlighting the residual module and the skip connection path.

Language Intermediate Level Attack (LILA)

Instead of relying solely on final output gradients, LILA utilizes intermediate representations. It replaces the gradient at a specific layer with a "directional guide" based on the difference between hidden states of adversarial and benign prompts. This method specifically targets the last token of the input prompt, which acts as a bottleneck for steering the model toward a jailbroken response.

Experimental Results

The combination of these methods, dubbed GCG-LSGM-LILA, was evaluated against benchmarks like AdvBench and HarmBench on models including Llama-2-7B/13B-Chat and Mistral-7B-Instruct.

Key findings include:

  • Match Rate Improvement: On Llama-2-7B-Chat, the match rate for exact target strings jumped from 54% to 87%.
  • Attack Success Rate (ASR): Query-specific ASR increased from 38% to 68%.
  • Universal Attacks: When attacking multiple prompts with a single suffix, average ASR rose from 26.68% to 60.32%.
  • Efficiency: These gains were achieved with zero additional computational cost. Furthermore, the researchers demonstrated that the candidate set size could be reduced to run the attack 25x faster while still outperforming the original GCG baseline.

Conclusion

By de-emphasizing "noisy" gradients from residual modules and focusing on critical bottleneck tokens, these methods significantly enhance the success of adversarial attacks. These insights suggest that understanding information flow through transformer layers is key to solving discrete optimization problems, with implications for both red-teaming and efficient prompt tuning.

Summary updated

BibTeX
@inproceedings{li2024improved,
  author = {Qizhang Li and Yiwen Guo and Wangmeng Zuo and Hao Chen},
  title = {{Improved Generation of Adversarial Examples Against Safety-aligned LLMs}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=8hBc843g1p},
}
Figure