NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks

Javad Rafiei Asl, Sidhant Narula, Mohammad Ghasemigol, Eduardo Blanco, Daniel TakabiEMNLP 2025

TL;DR

Large Language Models (LLMs) are increasingly vulnerable to "jailbreak" attacks, where malicious prompts bypass safety filters. This paper introduces NEXUS, a novel framework that systematically explores, refines, and executes multi-turn jailbreak attacks, achieving significantly higher success rates than existing methods by building a semantic network of potential attacks and iteratively refining them through simulated interactions.

What Problem Does This Paper Address?

LLMs, despite advancements in safety alignment, remain susceptible to jailbreak attacks. A particularly concerning type is multi-turn jailbreaks, which strategically embed harmful intent across seemingly benign conversations, making them harder to detect than single-turn attacks. Existing methods for crafting these attacks often lack systematic exploration of the adversarial space, rely on manual heuristics, or struggle with efficient query refinement. This paper aims to address these limitations by proposing a robust, automated framework for discovering and executing sophisticated multi-turn jailbreaks.

Methods

NEXUS is a modular, LLM-agnostic framework composed of three key phases:

  1. ThoughtNet: This phase constructs a semantic network of thought that represents the adversarial space. It hierarchically expands a harmful user goal into topics, contextual samples, and relevant entities. This structured approach ensures comprehensive coverage of potential attack vectors, moving beyond narrow subspaces.
  2. Feedback-driven Simulator: This component emulates real-time attack dynamics. It uses a "attacker-victim-judge" LLM collaboration to iteratively refine and prune candidate query chains. Harmfulness and semantic similarity benchmarks are used to guide this refinement process, discarding ineffective paths and optimizing promising ones.
  3. Network Traverser: In the final phase, this component executes the optimized query chains in real-time. It adaptively navigates the refined network to discover efficient and stealthy multi-turn jailbreaks against target LLMs.

Datasets

The paper evaluates NEXUS on two datasets:

  • HarmBench (Mazeika et al., 2024): A comprehensive evaluation suite with diverse harmful user intentions and standard implementations of black-box and white-box attacks.
  • AdvBench (Zou et al., 2023): A curated adversarial benchmark designed to assess LLM safety by probing susceptibility to a broad spectrum of harmful queries, including zero-shot and multi-turn jailbreaks.

The paper also mentions using specific LLM variants for defense-aware mitigation experiments, including X-Boundary-LLaMA-3-8B-Adapter, X-Boundary-Mistral-7B-Instruct-Adapter, LLaMA-Guard-3-8B, and Circuit Breakers-LLaMA-3-8B/Mistral-7B.

Key Insights and Ideas

  • Structured Adversarial Space: The ThoughtNet's hierarchical semantic network is a key innovation. Instead of random exploration, it systematically maps out potential attack paths, leading to more comprehensive discovery.
  • Iterative Refinement is Crucial: The Simulator's feedback-driven approach, using attacker-victim-judge LLMs, is vital for optimizing query chains. This iterative process allows for adaptive refinement that manual or heuristic methods struggle to achieve.
  • Generalizability Across Models: NEXUS demonstrates strong performance across a wide range of both closed-source (GPT-4o, Claude 3.5 Sonnet) and open-source (LLaMA-3-8B, Mistral-7B, Gemma-2-9B) LLMs, highlighting its model-agnostic nature.
  • Diversity in Attacks: NEXUS generates more diverse attack strategies compared to baselines, suggesting it can explore a broader range of vulnerabilities.

Key Results

  • High Attack Success Rate (ASR): NEXUS consistently outperforms state-of-the-art methods. For example, it achieved 94.8% ASR on GPT-40, surpassing ActorAttack by 10.3%. On LLaMA-3-8B-Instruct, it achieved 98.4% ASR, outperforming ActorAttack by +19.4%.
  • Improved Performance on Defense-Aware Models: Even when faced with advanced safety interventions like X-Boundary, LLaMA Guard 3, and Circuit Breakers, NEXUS maintained a strong performance, demonstrating its robustness.
  • Efficiency Gains: NEXUS shows improvements in average attack duration, API calls, and total queries compared to baselines like Crescendo and ActorAttack, making it more practical for real-world testing.
  • Optimal Parameters: Ablation studies identified 10 as the optimal number of main topics for ThoughtNet to balance diversity and efficiency, and a semantic alignment threshold of 0.15 for the Simulator.

Limitations and Future Work

  • Simulation Overhead: The feedback-driven simulation process can be computationally intensive, taking 15-30 minutes per input query, which limits scalability for high-throughput applications.
  • Early-Stage Query Chain Quality: For some specialized harmful intents, the initial query chains generated might lack sufficient harmfulness, requiring evaluation of additional chains and incurring extra latency.
  • Future Work: The authors suggest exploring more efficient simulation mechanisms and further investigating how to improve the quality of early-stage query chains for niche harmful intents. The framework's generalizability also opens avenues for developing more adaptive and resilient LLM defenses.

Summary updated

BibTeX
@inproceedings{asl2025nexus,
  author = {Javad Rafiei Asl and Sidhant Narula and Mohammad Ghasemigol and Eduardo Blanco and Daniel Takabi},
  title = {{NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure