Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models

Lang Gao, Xiangliang Zhang, Preslav Nakov, Xiuying Chen

Abstract

Despite rigorous safety alignment through Reinforcement Learning from Human Feedback (RLHF), Large Language Models (LLMs) remain susceptible to jailbreak attacks—carefully engineered prompts designed to bypass internal safety filters. This paper investigates the internal mechanisms of these attacks using a large-scale dataset of over 30,000 samples. We demonstrate that jailbreaks function by shifting internal activations into a benign-looking latent space. Based on these findings, we introduce Activation Boundary Defense (ABD), a lightweight mechanism that penalizes outlier activations. ABD achieves a 98% defense success rate with less than a 2% impact on the model's general intelligence.

Introduction

Jailbreak prompts, such as those providing instructions for illegal acts, exploit vulnerabilities in LLM safety alignment. While previous research has attempted to characterize these vulnerabilities, findings have been inconsistent regarding which layers are responsible for safety and whether the transition from harmful to benign intent is gradual or abrupt. These contradictions often stem from small-scale studies (typically ~100 samples). This research provides a definitive map of the jailbreak mechanism by scaling the analysis 300x, revealing how attacks manipulate the model's internal "safety boundary."

Methodology: Mapping the Safety Boundary

The authors conducted a massive analysis across seven jailbreak methods (including GCG, AutoDAN, and PAIR). Using Multi-Dimensional Scaling (MDS), they projected high-dimensional internal activations into 2D space to visualize the model's "thoughts."

MDS visualization of internal activations across layers 4, 15, and 30, showing the evolution of the safety boundary and how jailbreak prompts shift toward benign regions.

As shown in the visualization, harmful activations naturally cluster together, defining a "safety boundary." Successful jailbreaks work by "pushing" these activations out of the harmful cluster and into the space reserved for benign queries. Notably, this shift occurs early, with layers 5–15 acting as critical gatekeepers for identifying intent.

Activation Boundary Defense (ABD)

To counter this mechanism, the authors proposed Activation Boundary Defense (ABD). Unlike previous defenses that might "lobotomize" a model by broadly suppressing activations, ABD applies a smooth penalty function (using a tanh curve) specifically to outlier activations. This approach leaves benign activations untouched while "squashing" jailbreak-induced outliers back toward the safety zone. Bayesian Optimization was employed to automatically select the most effective layers and hyperparameters for each model, ensuring a balance between security and performance.

Key Results and Effectiveness

ABD demonstrates superior performance across multiple benchmarks:

  • High Defense Success Rate (DSR): ABD achieved an average DSR of over 98% against both seen and unseen attack styles.
  • Minimal Intelligence Tax: While other defenses (like IA) can drop general performance by over 30%, ABD maintains general capabilities with less than a 2% impact on benchmarks like Just-Eval.
  • Efficiency: The defense is computationally lightweight, adding less than 0.1 seconds of latency per query with $O(1)$ extra complexity.
Inclusion ratio across layers, demonstrating how ABD effectively keeps jailbreak activations within the safety boundary compared to baseline attacks.

The effectiveness of ABD is clearly visible in the inclusion ratio metrics, where it successfully retains activations within the defined safety boundary through the deeper layers of the model, whereas standard jailbreak attacks successfully escape this boundary.

Discussion and Future Work

This research suggests that the safety boundary in LLMs is more akin to a "safe zone" or sphere rather than a linear wall. Jailbreaks succeed by escaping this zone into unmonitored latent space. While ABD is highly effective for one-shot jailbreaks, future work is needed to address multi-turn dialogues where harmful intent may be introduced gradually. Additionally, the defense relies on the model having a baseline level of alignment; "under-aligned" models with fuzzy internal boundaries remain a challenge for boundary-based enforcement.

Summary updated

BibTeX
@inproceedings{gao2025shaping,
  author = {Lang Gao and Xiangliang Zhang and Preslav Nakov and Xiuying Chen},
  title = {{Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure