Introduction
As Large Language Models (LLMs) become increasingly integrated into daily applications, developers have implemented rigorous safety mechanisms, such as Reinforcement Learning from Human Feedback (RLHF), to prevent the generation of harmful content. However, most existing adversarial attacks rely on explicit "jailbreak" prompts or universal patterns that are easily identified and mitigated. This paper introduces Imposter.AI, a novel framework that exploits a critical vulnerability: hidden intentions. By decomposing a malicious goal into a series of seemingly benign sub-questions, Imposter.AI mimics human conversational strategies to bypass safety alignments in state-of-the-art models like GPT-4.
Methodology
The Imposter.AI pipeline is an automated, multi-stage process designed to obfuscate malicious intent through multi-turn dialogue. The process consists of five key stages:
- Knowledge Extraction: An uncensored LLM is used to generate a high-level roadmap (keypoints) for answering a harmful question.
- Harmful Question Decomposition: An "Agent LLM" (e.g., GPT-4) transforms these keypoints into a sequence of sub-questions, sorted from most benign to most sensitive to "prime" the target model.
- Question Toxicity Reduction: Sub-questions are rephrased using techniques such as Perspective Change (e.g., asking about historical criminal methods) and Intent Reversion (e.g., asking how to prevent a harm to indirectly learn how to cause it).
- Response Harmfulness Enhancement: The attack prompts for "Fictional Scenarios" or "Historical Examples" to ensure the target LLM provides detailed, actionable information.
- Summarization: Finally, the target LLM is asked to summarize the entire conversation into a step-by-step manual, consolidating fragmented data into a single harmful response.
Experimental Setup
The researchers evaluated Imposter.AI using the HarmfulQ dataset, which contains 200 explicit harmful questions across 16 categories, including terrorism and weapons. The framework was tested against GPT-3.5-turbo, GPT-4, and Llama2-13b, and compared to established baselines such as AIM (a manual jailbreak) and GCG (an optimization-based attack).
Results and Analysis
Imposter.AI demonstrated significant effectiveness across multiple models, particularly outperforming baselines on GPT-4.
- Superior Performance: On GPT-4, Imposter.AI achieved a Harmfulness score of 4.38/5 and an Executability score of 3.14/5, significantly higher than GCG and AIM.
- Persistence: Unlike manual jailbreaks like AIM, which saw a sharp decline in effectiveness as OpenAI patched its filters, Imposter.AI's success rate remained stable, highlighting the difficulty of filtering natural conversational strategies.
- The Usability-Security Trade-off: While Llama2-13b showed high resistance to the attack, it often did so through "over-refusal," rejecting even benignly rephrased versions of questions. This suggests that extreme safety alignments can significantly degrade a model's utility for legitimate users.
Conclusion
The success of Imposter.AI reveals that current LLM safety filters suffer from "contextual blindness," evaluating safety on a per-turn basis rather than analyzing the global intent of a conversation. To counter such multi-turn adversarial strategies, future defense mechanisms must evolve to understand the overarching goals of a dialogue history. The findings also underscore the need for a better balance between model safety and helpfulness.
Summary updated