Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles

Xiongtao Sun, Deyue Zhang, Dongdong Yang, Quanchen Zou, Hui LiarXiv 2024 · OpenReview

TL;DR

This paper introduces Contextual Fusion Attack (CFA), a novel black-box jailbreak attack method for Large Language Models (LLMs) that leverages multi-turn dialogues. CFA effectively circumvents LLM safety mechanisms by strategically embedding malicious intent within contextual scenarios, leading to significantly higher success rates and more harmful outputs compared to existing methods.

What Problem Does This Paper Address?

Large Language Models (LLMs) have become incredibly powerful, but their security vulnerabilities, particularly to "jailbreak" attacks, are a growing concern. These attacks trick LLMs into generating harmful, biased, or otherwise undesirable content, bypassing their safety guardrails. While single-turn attacks have been studied, multi-turn attacks, which involve a series of conversational turns, present unique challenges. Existing multi-turn methods often fail to fully exploit the conversational context, leading to semantic deviations and less effective attacks. This paper addresses the gap in understanding and exploiting the inherent advantages of multi-turn dialogues for more sophisticated and stealthy jailbreak attacks.

Methods

The core of this paper is the Contextual Fusion Attack (CFA) method. CFA operates in three stages:

  1. Preprocess: The attack target is analyzed to filter and extract key malicious keywords. This stage focuses on identifying the core harmful intent without directly exposing it.
  2. Context Generation: Based on the extracted keywords, contextual scenarios are generated. This involves creating a narrative or situation that indirectly relates to the malicious keywords, using prompt engineering techniques. The goal is to build a rich, coherent dialogue that supports the attack.
  3. Target Trigger: The generated context is integrated with the attack target. Crucially, the extracted malicious keywords are strategically replaced with semantically related but less overtly harmful terms within the context. This "dynamic loading" of malicious objectives, combined with the contextual framing, aims to bypass the LLM's security mechanisms without triggering direct refusals.

Datasets

The paper evaluates CFA on three widely used datasets for assessing jailbreak attacks:

  • Advbench: Contains 520 malicious prompts categorized into six types, including computer crimes and fraud.
  • MaliciousInstruct: Features 100 prompts covering ten distinct malicious intents like psychological manipulation and cyberbullying.
  • Jailbreakbench: Includes 100 data points covering a broad range of harmful behaviors, from violent content to privacy invasion.

The authors note that they classified Advbench into six categories, and Jailbreakbench covers 18 AdvBench behaviors, 27 TDC/HarmBench behaviors, and 55 unique behaviors from JBB-Behaviors.

Key Insights and Ideas

  • Multi-turn context is a powerful, underutilized weapon: The paper argues that multi-turn dialogues offer a significantly richer space for attack strategies than previously realized, providing crucial support for embedding malicious intent subtly.
  • Stealth through semantic fusion: CFA's innovation lies in its ability to replace direct malicious keywords with semantically similar but less obvious ones within a generated context. This "fusion" makes the attack harder to detect by security filters.
  • Dynamic loading of objectives: The attack's maliciousness isn't upfront but is "dynamically loaded" through the conversational flow, making it more akin to a runtime exploit than a static prompt.
  • LLM security alignment is weaker in complex contexts: The authors posit that the scarcity of long-text, multi-turn secure alignment datasets makes LLMs more vulnerable in these complex conversational scenarios.

Key Results

CFA demonstrated superior performance across various metrics and models:

  • Higher Success Rate: CFA achieved significantly higher average attack success rates (ASR) compared to baseline methods like PAIR, COU, and COA on multiple LLMs, including Llama3 and GPT-4. For instance, CFA achieved a 21% success rate on Llama3, doubling the attack success rate of other methods.
  • Increased Harmfulness: The outputs generated by CFA were found to be more toxic and harmful, as assessed by the Google Perspective API. This indicates that CFA not only bypasses defenses but also elicits more severe undesirable responses.
  • Superior Attack Consistency: CFA showed better semantic consistency, meaning the attack rounds remained more closely aligned with the original malicious intent, reducing semantic deviation and false positives often seen in other multi-turn attacks.
  • Robustness across Models: CFA exhibited stable attack effectiveness across different LLMs, highlighting its generalizability and the potential for widespread application.

Limitations and Future Work

  • Vicuna Model Issues: In experiments with the Vicuna model, issues like output repetition and chaotic generation were observed, which somewhat reduced CFA's success rate. The authors attribute this to inherent functional issues within the Vicuna model itself rather than a flaw in CFA's secure alignment.
  • Future Work: The paper suggests that this work lays the groundwork for further research into multi-turn attacks and aims to contribute to strengthening LLM robustness against such sophisticated threats. The authors' contribution is to elucidate the advantages of multi-turn attacks, paving the way for future research in this domain.

Summary updated

Abstract

Large language models (LLMs) have significantly enhanced the performance of numerous applications, from intelligent conversations to text generation. However, their inherent security vulnerabilities have become an increasingly significant challenge, especially with respect to jailbreak attacks. Attackers can circumvent the security mechanisms of these LLMs, breaching security constraints and causing harmful outputs. Focusing on multi-turn semantic jailbreak attacks, we observe that existing methods lack specific considerations for the role of multiturn dialogues in attack strategies, leading to semantic deviations during continuous interactions. Therefore, in this paper, we establish a theoretical foundation for multi-turn attacks by considering their support in jailbreak attacks, and based on this, propose a context-based contextual fusion black-box jailbreak attack method, named Context Fusion Attack (CFA). This method approach involves filtering and extracting key terms from the target, constructing contextual scenarios around these terms, dynamically integrating the target into the scenarios, replacing malicious key terms within the target, and thereby concealing the direct malicious intent. Through comparisons on various mainstream LLMs and red team datasets, we have demonstrated CFA's superior success rate, divergence, and harmfulness compared to other multi-turn attack strategies, particularly showcasing significant advantages on Llama3 and GPT-4.
BibTeX
@misc{sun2024multi,
  author = {Xiongtao Sun and Deyue Zhang and Dongdong Yang and Quanchen Zou and Hui Li},
  title = {{Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles}},
  year = {2024},
  eprint = {2408.04686},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.04686},
}
Figure