Distillability of LLM Security Logic: Predicting Attack Success Rate of Outline Filling Attack via Ranking Regression

Tianyu Zhang, Zihang Xi, Jingyu Hua, Sheng Zhong

Abstract

In the realm of black-box jailbreak attacks on large language models (LLMs), the feasibility of constructing a narrow safety proxy, a lightweight model designed to predict the attack success rate (ASR) of adversarial prompts, remains underexplored. This work investigates the distillability of an LLM's core security logic. We propose a novel framework that incorporates an improved outline filling attack to achieve dense sampling of the model's security boundaries. Furthermore, we introduce a ranking regression paradigm that replaces standard regression and trains the proxy model to predict which prompt yields a higher ASR. Experimental results show that our proxy model achieves an accuracy of 91.1 percent in predicting the relative ranking of average long response (ALR), and 69.2 percent in predicting ASR. These findings confirm the predictability and distillability of jailbreak behaviors, and demonstrate the potential of leveraging such distillability to optimize black-box attacks.

Introduction

Jailbreaking Large Language Models (LLMs)—the process of bypassing safety filters to generate harmful content—is typically conducted in a "black-box" setting. Attackers often rely on trial and error because they lack access to the model's internal weights. While proxy models are frequently used to assist in these attacks, they generally focus on mimicking the target model's content generation rather than its security boundaries.

This research addresses the challenge of building a Narrow Safety Proxy: a lightweight model specifically trained to predict the Attack Success Rate (ASR) of adversarial prompts. The development of such a proxy faces two primary obstacles:

  1. Sampling Sparsity: Most jailbreak attempts result in complete refusal, providing little information about the precise location of the security boundary.
  2. Domain Shift: LLMs exhibit varying sensitivities across different harmful categories (e.g., "suicide" vs. "software piracy"), making absolute ASR prediction difficult.

Methodology

The Outline Filling Attack

To address sampling sparsity, the authors developed the Outline Filling Attack. This method uses an auxiliary model (e.g., GPT-3.5-turbo) to decompose a harmful request into a benign-looking, hierarchical outline. By asking the target LLM to simply "fill in the contents" of this outline, the attack bypasses holistic intent detection. This allows for the generation of numerous semantically similar but structurally distinct prompts for a single topic, effectively probing the model's probabilistic boundaries.

Example of an Outline Filling Attack prompt, where a harmful request is decomposed into a structural outline to bypass safety filters.

Ranking Regression and Surrogate Model Construction

Instead of attempting to predict absolute ASR values, which are prone to domain shift, the framework utilizes a Ranking Regression paradigm. The proxy model is trained on pairs of prompts to determine which one is more likely to succeed. This relative approach is more robust across different harmful domains. To convert these pairwise comparisons into a global "Jailbreak Success Score," the researchers employed the Bradley-Terry-Luce (BTL) model.

The comprehensive workflow for constructing the surrogate model and executing the automatic IRA attack.

Experimental Setup

The study utilized 80 dangerous questions from the JailbreakBench (JBB) and HarmBench (HB) datasets. For each question, 75 distinct "Outline Filling" prompts were generated and tested 20 times each against three target models: Llama-3-8B, GPT-4o-mini, and Qwen. This extensive data collection provided the necessary interaction behaviors to train the Llama-3-8B based proxy model.

Results

The experimental results demonstrate that LLM security logic is indeed distillable:

  • Predictive Accuracy: The proxy achieved 91.1% accuracy in predicting the relative ranking of Average Long Response (ALR), which measures whether a model provides a substantive response versus a refusal.
  • ASR Prediction: For the more complex task of predicting actual harmfulness (ASR), the proxy achieved 69.2% accuracy.
  • Attack Optimization: By using the proxy to select top-ranked prompts, the researchers reduced the First Attack Success Cost (FASC) by over 70%. In some instances, the number of queries required to find a successful jailbreak dropped from 60 to fewer than 17.

Discussion and Limitations

The success of the Outline Filling Attack suggests that LLM safety filters are vulnerable to structural fragmentation, where harmful concepts are broken into seemingly benign sub-tasks. Furthermore, the ability of a small 8B-parameter model to learn the security preferences of larger models indicates that safety alignment logic can be "stolen" to optimize attacks.

However, the study has limitations. It focused primarily on structural splitting attacks; the effectiveness of this proxy approach for other techniques like code injection or role-playing remains to be seen. Additionally, while the proxy was tested against models like Llama-3 and GPT-4o-mini, further validation against the largest frontier models is necessary. Future work could also explore defensive applications, such as "Anti-Proxy" filters designed to detect prompts engineered to probe security boundaries.

Summary updated

Abstract

In the realm of black-box jailbreak attacks on large language models (LLMs), the feasibility of constructing a narrow safety proxy, a lightweight model designed to predict the attack success rate (ASR) of adversarial prompts, remains underexplored. This work investigates the distillability of an LLM's core security logic. We propose a novel framework that incorporates an improved outline filling attack to achieve dense sampling of the model's security boundaries. Furthermore, we introduce a ranking regression paradigm that replaces standard regression and trains the proxy model to predict which prompt yields a higher ASR. Experimental results show that our proxy model achieves an accuracy of 91.1 percent in predicting the relative ranking of average long response (ALR), and 69.2 percent in predicting ASR. These findings confirm the predictability and distillability of jailbreak behaviors, and demonstrate the potential of leveraging such distillability to optimize black-box attacks.
BibTeX
@misc{zhang2025distillability,
  author = {Tianyu Zhang and Zihang Xi and Jingyu Hua and Sheng Zhong},
  title = {{Distillability of LLM Security Logic: Predicting Attack Success Rate of Outline Filling Attack via Ranking Regression}},
  year = {2025},
  eprint = {2511.22044},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.22044},
}
Figure