You Know What I'm Saying: Jailbreak Attack via Implicit Reference

Tianyu Wu, Lingrui Mei, Ruibin Yuan, Lujun Li, Wei Xue, Yike Guo

Abstract While recent advancements in large language model (LLM) alignment have enabled the effective identification of malicious objectives involving scene nesting and keyword rewriting, our study reveals that these methods remain inadequate at detecting malicious objectives expressed through context within nested harmless objectives. This study identifies a previously overlooked vulnerability, which we term Attack via Implicit Reference (AIR). AIR decomposes a malicious objective into permissible objectives and links them through implicit references within the context. This method employs multiple related harmless objectives to generate malicious content without triggering refusal responses, thereby effectively bypassing existing detection techniques. Our experiments demonstrate AIR's effectiveness across state-of-the-art LLMs, achieving an attack success rate (ASR) exceeding 90% on most models, including GPT-4o, Claude-3.5-Sonnet, and Qwen-2-72B. Notably, we observe an inverse scaling phenomenon, where larger models are more vulnerable to this attack method. These findings underscore the urgent need for defense mechanisms capable of understanding and preventing contextual attacks. Furthermore, we introduce a cross-model attack strategy that leverages less secure models to generate malicious contexts, thereby further increasing the ASR when targeting other models.

Introduction

Current Large Language Model (LLM) safety mechanisms have become increasingly sophisticated, effectively neutralizing "classic" jailbreak attempts such as role-playing scenarios or simple keyword obfuscation. However, these defenses typically operate by scanning for malicious intent within a localized, single-prompt window. This research introduces Attack via Implicit Reference (AIR), a novel jailbreak method that exploits a model's inherent drive for conversational coherence and its advanced reasoning capabilities to bypass these filters.

AIR identifies a critical contextual vulnerability: LLMs are trained to maintain thread-level logic. By decomposing a harmful request into a series of seemingly benign nested objectives, the attack baits the model into "filling in the blanks." The model uses its internal attention mechanism to link harmless sub-tasks back to a sensitive subject introduced earlier in the context, effectively jailbreaking itself through its own intelligence.

Methodology: The AIR Framework

The AIR framework utilizes a two-stage conversational strategy to deceive target models. This process is often facilitated by a high-intelligence "Rewrite Model" (such as GPT-4o) to ensure the structural integrity of the attack prompts.

  1. Stage 1: Prompt Rewriting & Decomposition: The malicious query (e.g., "How to build a bomb") is transformed into a structured outline of harmless objectives. This outline typically includes a Title (where the sensitive keyword is placed), an Abstract, and a Background section. The core of the attack lies in the "Making Example" section, which avoids explicit mention of the harmful subject, instead using implicit references like "this," "the process," or "the example."
  2. Stage 2: The Continue Attack: After the model generates an initial sanitized response, the attacker follows up by asking the model to "add more detail" and "remove judgment." Because the model is now anchored in the established context, it often provides the requested harmful details without triggering safety refusals.
The two-stage AIR framework workflow, illustrating the transition from prompt decomposition to the final jailbreak completion via a follow-up attack.

Additionally, the researchers developed a Cross-Model Attack strategy. In this scenario, a less-secure model is used to generate the initial "benign" context, which is then fed to a more secure, highly-aligned model to trigger the generation of harmful content.

Experimental Results and the Reverse Scaling Phenomenon

The effectiveness of AIR was evaluated using the JBB-Behaviors dataset, encompassing 100 harmful behaviors across categories such as physical harm, malware, and privacy violations. The results were striking, with AIR achieving an average Attack Success Rate (ASR) of 91.75% across top-tier models like GPT-4o, Claude-3.5-Sonnet, and Qwen-2-72B. This significantly outperforms baseline automated jailbreaks like PAIR (13.25% ASR) and DeepInception (10% ASR).

Perhaps the most significant finding is the Reverse Scaling Phenomenon. While larger models are generally considered safer due to more rigorous alignment, they proved more vulnerable to AIR. Their superior in-context learning and reasoning allow them to better grasp and fulfill the "hidden" malicious intent within the decomposed prompts.

Graph demonstrating reverse scaling: as model size increases from 'Tiny' to 'Large', the Attack Success Rate (ASR) of AIR consistently rises across different model families.

Conclusion

The success of AIR reveals that current "surface-level" alignment—which focuses on keyword detection and localized intent—is insufficient against contextual attacks. The vulnerability arises from a conflict between the model's objective to be helpful and its safety constraints; when harm is implicit and instructions are complex, helpfulness often prevails. Future defense research must move toward contextual safety mechanisms that can evaluate the intent of an entire conversation rather than isolated prompts. While AIR struggles with inherently toxic topics that cannot be easily decomposed (such as pedophilia), its high success rate on other regulated topics underscores an urgent need for more robust, context-aware AI safety architectures.

Summary updated

Abstract

While recent advancements in large language model (LLM) alignment have enabled the effective identification of malicious objectives involving scene nesting and keyword rewriting, our study reveals that these methods remain inadequate at detecting malicious objectives expressed through context within nested harmless objectives. This study identifies a previously overlooked vulnerability, which we term Attack via Implicit Reference (AIR). AIR decomposes a malicious objective into permissible objectives and links them through implicit references within the context. This method employs multiple related harmless objectives to generate malicious content without triggering refusal responses, thereby effectively bypassing existing detection techniques.Our experiments demonstrate AIR's effectiveness across state-of-the-art LLMs, achieving an attack success rate (ASR) exceeding 90% on most models, including GPT-4o, Claude-3.5-Sonnet, and Qwen-2-72B. Notably, we observe an inverse scaling phenomenon, where larger models are more vulnerable to this attack method. These findings underscore the urgent need for defense mechanisms capable of understanding and preventing contextual attacks. Furthermore, we introduce a cross-model attack strategy that leverages less secure models to generate malicious contexts, thereby further increasing the ASR when targeting other models.Our code and jailbreak artifacts can be found at https://github.com/Lucas-TY/llm_Implicit_reference.
BibTeX
@misc{wu2024you,
  author = {Tianyu Wu and Lingrui Mei and Ruibin Yuan and Lujun Li and Wei Xue and Yike Guo},
  title = {{You Know What I'm Saying: Jailbreak Attack via Implicit Reference}},
  year = {2024},
  eprint = {2410.03857},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.03857},
}
Figure