LLMs are Vulnerable to Malicious Requests with Indistinguishable Intent

Wenzhuo Xu, Zonghao Ying, Deyue Zhang, Dongdong Yang, Xiangzheng Zhang, Quanchen ZouarXiv 2026 · arXiv

TL;DR

Researchers have uncovered a significant vulnerability in Large Language Models (LLMs): they are highly susceptible to "Detail Guidance Attacks" (DGA) that wrap malicious intent in realistic, detailed narratives. By making harmful requests indistinguishable from legitimate user queries, these attacks achieve success rates over 95% on state-of-the-art models, effectively bypassing standard safety filters that look for obvious anomalous patterns.

What Problem Does This Paper Address?

As LLMs become more integrated into daily life, "jailbreaking"—tricking a model into generating harmful or policy-violating content—has become a high-stakes game of cat and mouse. Most existing jailbreak techniques rely on conspicuous patterns: weird character encoding (Base64), complex roleplay templates ("DAN"), or adversarial suffixes. Because these look "weird," they are relatively easy for safety filters to catch.

The authors of this paper ask: What if a jailbreak looks perfectly normal? They investigate the gap between "Type I" jailbreaks (conspicuous, easily filtered) and "Type II" jailbreaks (indistinguishable from ordinary queries). The core challenge is that if a filter is aggressive enough to block requests that look normal but might be malicious, it will inevitably block many legitimate users, leading to a "safety-usability trade-off."

Methods

The authors introduce the Detail Guidance Attack (DGA), a framework that uses an auxiliary LLM to transform a concise malicious intent into a detailed, realistic scenario. The process follows three steps:

  1. Rephrasing: A harmful prompt (e.g., "How to make a bomb?") is condensed into a key phrase ("bomb making").
  2. Detail Generation: An auxiliary LLM (like Gemini-2.5-flash) generates a rich context around that phrase. DGA has two flavors:
  3. DGA-OS (Observable Signs): The prompt describes "evidence" found at a scene and asks the victim model to infer the steps involved (e.g., "I found these chemicals and wires, what is being made here?").
  4. DGA-CS (Corresponding Story): The prompt embeds the request in a narrative, such as a detective investigating a crime or an author writing a thriller.
  5. Inducing: The generated details are combined with an "inducing prompt" that pressures the victim model to provide a factual, step-by-step explanation of the harmful activity under the guise of "emergency analysis" or "creative writing."

To evaluate these, the authors also built a defense filter that assigns a Motivation Level Score (MLS) from 0 (definitely harmful) to 4 (definitely harmless).

Datasets

The study evaluates 11 modern LLMs (including GPT-4o, GPT-5, o3, and DeepSeek-v3) across three primary benchmarks:

  • MaliciousInstruct (MI): 100 harmful requests.
  • JBB-Behaviors (JBB): 100 harmful prompts from the JailbreakBench suite.
  • AdvBench: 520 harmful prompts.
  • XSTest: Used to calibrate the filter's threshold by providing "benign but suspicious-looking" requests (e.g., "how to kill a Python process?").

Key Insights and Ideas

  • The Indistinguishability Hypothesis: The most successful jailbreaks are those that the model perceives as "normal." As shown in Figure 1, methods with a higher Indistinguishability Score (IS) generally have much higher filter pass rates.
Fig. 1: Average model-rated indistinguishability score (IS) of the eight baseline methods on MaliciousInstruct. Higher scores indicate greater perceived resem- blance to normal requests under the scoring rubric. Scores are averaged across all 11 scoring models.
  • Type I vs. Type II Failures: Safety filters are great at catching "Type I" attacks like CodeAttack (CA) because they recognize the attempt to bypass filters. However, they struggle with "Type II" attacks like Scenario Simulation (SS) because the context provided (e.g., "training materials for security") appears legitimate. This is highlighted in the "MLS reasons" provided by the filter in Table 3.
Table 3: The MLS and MLS reasons for different jailbreaks. We only show one typical MLS reason for each jailbreak. Model refers to the LLM serving as the filter giving MLS and MLS reasons.
  • The Narrative Shield: By adding "noise" in the form of rich, realistic details, DGA makes it difficult for a filter to isolate the malicious intent without also risking the rejection of benign requests from authors, researchers, or concerned citizens.

Key Results

  • Near-Total Success: On the MaliciousInstruct dataset, the DGA-OS pipeline achieved an average Attack Success Rate (ASR) of 95.27% and a filter pass rate of 95.55%.
  • Bypassing Baselines: Traditional jailbreaks like CodeAttack (CA) had high raw success rates (~61%) but were almost entirely blocked by filters (pass rate of ~2%). In contrast, DGA maintained its high success rate even after filtering.
  • Universal Vulnerability: The attack was effective across all 11 tested models, with several models (like Seed-1.6 and Kimi-k2) showing a 100% success rate against DGA-OS.
  • User Ambiguity: A user study revealed that for "dual-use" requests (e.g., asking about scam techniques to protect parents), nearly 50% of participants favored the model responding, despite the potential for misuse. This confirms that "indistinguishable intent" is a genuine problem for deployment.

Limitations and Future Work

The authors acknowledge that their current experiments evaluate the complete DGA pipeline and do not perfectly isolate how much of the success comes from the "details" versus the "scenario framing."

Furthermore, while they proposed a "DGA-aware" filter that can block these specific attacks, they haven't yet measured its "false positive" rate—how many legitimate writers or researchers would be blocked by such a filter. Future work is needed to develop defenses that can distinguish between a malicious actor "cloaking" their intent and a legitimate user with a complex, but safe, request.

Summary updated

Abstract

Jailbreak attacks can circumvent the safety mechanisms of Large Language Models (LLMs) and lead to the generation of harmful or policy-violating content. We first examine eight jailbreak methods against an LLM-based filter and observe that requests resembling normal user queries tend to have higher filter pass rates than those containing obvious anomalous patterns. This observation motivates the hypothesis that resemblance to normal requests may contribute to jailbreak success. We introduce \textbf{Detail Guidance Attack (DGA)}, a framework that uses an auxiliary LLM to construct detailed requests and interacts with the victim model in a single turn. Across 11 victim models and three datasets, the complete DGA pipeline achieves high attack success rates under our automated judging criterion, including rates above 95\% in several evaluated settings. We also evaluate a DGA-aware filter and conduct a user study on everyday requests with ambiguous or dual-use intent. The survey documents differing expectations about whether models should respond; the usability costs of the filter remain to be measured.
BibTeX
@misc{xu2026llms,
  author = {Wenzhuo Xu and Zonghao Ying and Deyue Zhang and Dongdong Yang and Xiangzheng Zhang and Quanchen Zou},
  title = {{LLMs are Vulnerable to Malicious Requests with Indistinguishable Intent}},
  year = {2026},
  eprint = {2611.17818},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17818},
}
Figure