Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models

Shuang Liang, Zhihao Xu, Jialing Tao, Hui Xue, Xiting Wang

TL;DR

This paper introduces "Learning to Detect" (LoD), a novel framework for identifying unknown jailbreak attacks in Large Vision-Language Models (LVLMs). LoD shifts from learning attack-specific patterns to learning general task-specific safety representations, enabling robust detection of novel attacks without prior exposure.

What Problem Does This Paper Address?

Large Vision-Language Models (LVLMs), which combine text and image understanding, have become increasingly powerful but also introduce new security vulnerabilities. A significant concern is "jailbreaking," where malicious prompts can trick these models into generating harmful or unsafe content. Existing methods for detecting these jailbreaks often struggle:

  • Attack-specific learning: Methods trained on known attacks fail to generalize to new, unseen attack techniques.
  • Heuristic-based methods: While they can detect unseen attacks, they often lack the accuracy and efficiency needed for real-world deployment.

This gap leaves LVLMs susceptible to serious safety risks, necessitating a more robust and generalizable detection approach.

Methods

The LoD framework comprises two key modules:

  1. Multi-modal Safety Concept Activation Vectors (MSCAV): This module focuses on representation learning. It leverages the internal activations of an LVLM to estimate the probability that the model considers an input unsafe across different layers. By aggregating these layer-wise probabilities, MSCAV creates a representation that filters out irrelevant information and captures safety-critical signals. This is achieved by training linear classifiers on safe and unsafe (but not attacked) inputs.
  2. Safety Pattern Auto-Encoder (SPAE): This module handles attack classification. It treats the detection of unknown attacks as an anomaly detection problem. SPAE is trained exclusively on MSCAVs derived from safe inputs. During testing, it reconstructs the MSCAVs of incoming prompts. High reconstruction errors indicate deviations from the learned "safe" patterns, flagging the input as potentially attacked. This approach avoids the need for attack-specific training data, promoting generalization.

Datasets

  • Training Data:
    • Unsafe inputs from AdvBench (Chen et al., 2022).
  • Safe inputs from GQA (Hudson and Manning, 2019).
  • Multimodal inputs were created by synthesizing images for both safe and unsafe texts using Pixart-Sigma (Chen et al., 2024).
  • Validation Data: Used for threshold selection.
  • Testing Data:
    • SEED-Bench (Li et al., 2024) for safe inputs.
  • MM-SafetyBench (Liu et al., 2024c) for unsafe inputs.
  • Specific attack datasets for MML, JOOD, HADES, VAJM, and UMK.
  • MM-Vet2 (Yu et al., 2024) was also used for supplementary evaluation.

Key Insights and Ideas

  • Shifting Focus from Attacks to Tasks: The core idea is that learning general safety concepts within the LVLM's internal representations is more effective for detecting unknown attacks than trying to learn specific attack patterns.
  • Internal Activations as Safety Signals: LVLMs' internal activations, particularly across multiple layers, contain rich information about safety risks, even without explicit labels for attacked inputs.
  • Anomaly Detection for Generalization: Framing unknown attack detection as anomaly detection, where the model learns what "safe" looks like and flags deviations, is key to generalizing to unseen threats.
  • Visual Modality Accelerates Safety Concept Emergence: The authors observe that visual information in LVLMs helps safety concepts emerge in earlier layers compared to text-only LLMs.

Key Results

  • High Detection Accuracy: LoD consistently achieves the highest AUROC scores across three LVLMs (LLaVA, Qwen2.5-VL, CogVLM) and five diverse jailbreak attack methods.
  • Significant Improvement: LoD improves the minimum AUROC across attacks by up to 62.31% and achieves average AUROC scores around 0.999 in many cases.
  • Efficiency: LoD is computationally efficient, reducing detection time by up to 62.7% compared to the best baselines.
  • Robustness: The method demonstrates robustness to hyperparameter choices, particularly the layer selection threshold.
  • Generalization: LoD significantly outperforms baselines, especially on unseen attacks, highlighting its strong generalization capabilities.

Limitations and Future Work

  • LVLM Specificity: The method's performance might depend on the specific LVLM architecture. Significant architectural changes or new safety alignment mechanisms might require re-adaptation.
  • Anticipating Future Attacks: While LoD is designed for generalization, it cannot fully anticipate future adaptive attacks that might specifically target internal representations in novel ways. Exploring such scenarios is a future research direction.
  • Hyperparameter Sensitivity: While generally robust, the paper notes that overly large layer selection thresholds might lead to minor performance degradation by removing informative layers.
  • Future Work: The authors suggest exploring LoD's applicability to other modalities and attack forms beyond jailbreaks.

Summary updated

Abstract

Despite extensive alignment efforts, Large Vision-Language Models (LVLMs) remain vulnerable to jailbreak attacks, posing serious safety risks. To address this, existing detection methods either learn attack-specific parameters, which hinders generalization to unseen attacks, or rely on heuristically sound principles, which limit accuracy and efficiency. To overcome these limitations, we propose Learning to Detect (LoD), a general framework that accurately detects unknown jailbreak attacks by shifting the focus from attack-specific learning to task-specific learning. This framework includes a Multi-modal Safety Concept Activation Vector module for safety-oriented representation learning and a Safety Pattern Auto-Encoder module for unsupervised attack classification. Extensive experiments show that our method achieves consistently higher detection AUROC on diverse unknown attacks while improving efficiency. The code is available at https://anonymous.4open.science/r/Learning-to-Detect-51CB.
BibTeX
@misc{liang2025learning,
  author = {Shuang Liang and Zhihao Xu and Jialing Tao and Hui Xue and Xiting Wang},
  title = {{Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models}},
  year = {2025},
  eprint = {2510.15430},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.15430},
}
Figure