TL;DR
This paper introduces Attack via Implicit Reference (AIR), a two-stage jailbreak technique that bypasses large language model (LLM) safety guardrails by decomposing harmful requests into individually benign, nested objectives connected only through implicit contextual references (e.g., pronouns like "this"). AIR achieves an attack success rate (ASR) exceeding 90% against frontier models (GPT-4o, Claude-3.5-Sonnet, LLaMA-3-70B), exposing an "inverse scaling" vulnerability where larger, more capable models are easier to jailbreak due to their superior in-context reasoning and coreference resolution.
What Problem Does This Paper Address?
Current safety alignment techniques (RLHF, DPO, adversarial fine-tuning) excel at catching direct malicious requests, obvious keyword obfuscation, and overt role-play/scenario nesting. The authors argue that safety alignment largely operates on the Superficial Alignment Hypothesis: models retain latent pre-training knowledge and only learn surface-level mapping to refuse specific malicious queries without restricting benign ones on the same topic.
Consequently, guardrails struggle when an adversarial intent is never stated explicitly in a single sentence or prompt, but instead emerges across a multi-part structure linked by implicit context. Existing defenses fail to distinguish benign contextual completion from an assembled malicious instruction.
Methods
AIR operates in a structured two-stage pipeline:
- Stage 1: Prompt Rewriting & Nested Objective Generation
- An automated rewriting model (e.g., GPT-4o) reformulates a malicious prompt into an academic or structured outline.
- Decomposition: The subject is isolated into a benign title (e.g., "Introduction to [Topic]"), while the harmful action is placed into a separate paragraph task that omits the topic name, using implicit references instead (e.g., "Provide an example tutorial about doing this").
- Competing Objectives: To distract the target model's refusal mechanisms, several additional innocuous tasks are injected into the prompt (e.g., Abstract, Background, History). The model's imperative to fulfill benign instructions competes directly with its latent refusal heuristics.
- Stage 2: Benign Refinement ("Continue Attack")
- The user issues a follow-up prompt requesting elaboration on the target paragraph while instructing the model to remove high-level disclaimers or judgment. Because this follow-up query contains zero explicit toxic keywords or overt malicious instructions, it smoothly clears standard input filters.
- Cross-Model Conversation Attack Strategy
- To attack stricter or more sensitive target models, attackers can execute Stage 1 using a less aligned, less sensitive model to generate the initial context, then feed that conversation history into the stricter model for Stage 2 refinement.
Datasets
- JBB-Behaviors Benchmark: 100 harmful behaviors from JailbreakBench (Chao et al., 2024), spanning 10 core safety policy violation categories (e.g., malware/hacking, harassment, privacy violations, and physical harm). This benchmark synthesizes test cases from AdvBench, HarmBench, and the Trojan Detection Challenge.
- Evaluators Used:
- GPT-4 Malicious Score Evaluator (1–10 scale).
- LLaMA-3-70B Jailbreak Evaluator (binary safe/unsafe classification).
- GCG string/pattern-matching evaluator to detect standard refusal phrasing.
Key Insights and Ideas
- Inverse / Reverse Scaling Vulnerability: Larger models are substantially more vulnerable to AIR than smaller models. Smaller models (such as Qwen-2-0.5B) often lack the attention capacity and contextual understanding needed to resolve implicit references, inadvertently blunting the attack. Conversely, advanced models excel at connecting implicit references across sections, leading to higher rates of harmful output generation.
- Refusal Rate vs. Execution Ability: The probability of a model explicitly refusing a prompt did not change significantly across model sizes; rather, larger models were simply more capable of generating the detailed, context-aware content required to complete the exploit.
- Objective Overload: Increasing the number of benign framing objectives ($K$) consistently increased attack success rates (e.g., increasing $K$ from 2 to 6 increased ASR from 64% to 83% on LLaMA-3-8B).
- Failure of Off-the-Shelf Guardrails: Common post-processing and filtering defenses (SmoothLLM, PerplexityFilter, Erase-and-Check, and SafeDecoding) were largely ineffective against AIR because the individual prompt inputs contain low perplexity and lack explicit toxic tokens. When defenses like SafeDecoding were tuned aggressively enough to block AIR, they exhibited severe over-refusal on harmless prompts.
Key Results
- Attack Success Rate (ASR): AIR achieved an average ASR of 91.75% across frontier models evaluated with the LLaMA-3-70B judge, dramatically outperforming prior baselines:
- GPT-4o: 95% ASR (vs. 30% for DeepInception, 83% for Past Tense).
- Claude-3.5-Sonnet: 94% ASR (vs. 0% for DeepInception, 27% for Past Tense, 2% for PAIR).
- LLaMA-3-70B: 88% ASR.
- Qwen-2-72B: 90% ASR.
- First Attack Success Rate (FASR): AIR achieved an average single-attempt FASR of 52% without requiring iterative prompt optimization turns.
- Cross-Model Attack: Seeding conversational history with GPT-4o boosted downstream ASR on harder targets (e.g., raising LLaMA-3-8B ASR from 77% to 81%, and Qwen-2-1.5B from 67% to 70%).
Limitations and Future Work
- Dependency on Frontier Rewriters: The attack framework requires a strong rewriting LLM (like GPT-4o-0513); older models (such as GPT-3.5-Turbo) fail to construct the nuanced outlines correctly, and some newer aligned models increasingly refuse the rewriting meta-prompt itself.
- Scope Constraints: The study primarily tested writing and document generation structures (though the authors note the concept generalizes to code class/function nesting or tabular formats).
- Undecomposable Harms: Certain extreme categories (such as non-consensual sexual content or severe physical violence) resist decomposition into benign sub-tasks, remaining consistently blocked by frontier models.
- Defense Outlook: Preliminary checks on reasoning models (such as OpenAI o1-preview) indicated that chain-of-thought verification during inference helped detect the underlying intent. The authors argue future alignment must integrate implicit coreference analysis during safety post-training rather than relying on shallow token-level filters.
Summary updated