JailbreakHunter: A Visual Analytics Approach for Jailbreak Prompts Discovery from Large-Scale Human-LLM Conversational Datasets

Zhihua Jin, Shiyi Liu, Haotian Li, Xun Zhao, Huamin Qu

Abstract

As Large Language Models (LLMs) proliferate, so do adversarial "jailbreak prompts" designed to bypass safety filters. While many prompts are documented publicly, a significant portion of successful attacks remain "private," buried within massive "in-the-wild" conversational datasets. We present JailbreakHunter, a visual analytics system that enables researchers to efficiently discover and analyze these hidden adversarial strategies. By integrating automated clustering with a multi-level interactive workflow—group, conversation, and turn-level—JailbreakHunter allows experts to navigate millions of logs to identify evolving jailbreak patterns.

Introduction

The rapid adoption of LLMs has been accompanied by the emergence of jailbreak prompts—inputs engineered to force models into generating prohibited content. Identifying these prompts in large-scale datasets like WildChat or LMSYS-Chat-1M is a "needle in a haystack" problem. Traditional moderation tools often fail to capture the nuances of multi-turn adversarial dialogues or the evolution of "zero-day" prompts that do not match known templates. There is a critical need for tools that support human-in-the-loop discovery of these sophisticated attack patterns.

The JailbreakHunter System

JailbreakHunter employs a model-agnostic, three-level visual analytics approach to bridge the gap between automated detection and human expertise.

The JailbreakHunter visual analytics pipeline, spanning from dataset storage to multi-level analysis.

Multi-Level Analysis Workflow

The system architecture facilitates a top-down exploration of conversational data through three distinct views:

  1. Group-level (Cluster View): Using Sentence Transformers and UMAP projection, the system maps conversations onto a 2D plane. Kernel Density Estimation (KDE) highlights data distribution, while the Attack Success Rate (ASR)—calculated via the OpenAI Moderation API—identifies "hotspots" of likely jailbreaks.
  2. Conversation-level (Conversation View): Users can drill down into specific clusters to examine multi-turn dialogues. Visual "thumbnails" represent conversation progression, flagging malicious turns and measuring similarity to a reference set of known jailbreak prompts.
  3. Turn-level (Comparison View): This view enables a granular comparison between suspicious prompts and known jailbreaks, highlighting keyword overlaps and semantic similarities to distinguish between known techniques and novel strategies.
The JailbreakHunter interface featuring (a) Filter Panel, (b) Cluster View, (c) Conversation View, and (d) Comparison View.

Key Insights and Results

Case studies with security researchers revealed several critical findings:

  • The "Private" Gap: Many successful attacks in the wild do not follow public templates, necessitating visual discovery tools.
  • Multi-turn Persistence: Models often succumb to repeated or "forcing" instructions over multiple turns, even if they initially reject the request.
  • Prefix Injection: The system identified patterns where users trick models by forcing them to start responses with specific disclaimers, which inadvertently bypasses safety logic.

Expert evaluations validated the system's effectiveness. Experts successfully identified new variants of jailbreak prompts not present in reference databases. On a 7-point Likert scale, experts rated the ease of identifying suspicious conversations at 6.33/7. The system achieved a System Usability Scale (SUS) score of 68.70, indicating good usability despite a steep learning curve.

Conclusion and Future Work

JailbreakHunter demonstrates the power of visual analytics in securing LLMs. While currently limited to text-only data and datasets under 10 million points, future work aims to support multi-modal inputs and integrate LLM-based summarization to further streamline the discovery process. By providing a "compass" through the ASR metric and semantic mapping, JailbreakHunter empowers researchers to stay ahead of evolving adversarial threats.

Summary updated

BibTeX
@article{jin2025jailbreakhunter,
  author = {Zhihua Jin and Shiyi Liu and Haotian Li and Xun Zhao and Huamin Qu},
  title = {{JailbreakHunter: A Visual Analytics Approach for Jailbreak Prompts Discovery from Large-Scale Human-LLM Conversational Datasets}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2025},
  doi = {10.1109/tvcg.2025.3557568},
}
Figure