TL;DR
This paper exposes a novel security vulnerability in text-to-image diffusion models equipped with Image Prompt Adapters (IP-Adapters). By exploiting open-source vision encoders, the authors demonstrate a "hijacking attack" where imperceptible adversarial perturbations added to clean images trick benign users into generating Not-Safe-For-Work (NSFW) content. This stealthy exploit effectively turns standard image prompts into Trojan horses, bypassing traditional text-based filters and shifting blame for explicit outputs onto the service providers.
What Problem Does This Paper Address?
Most existing jailbreak attacks on text-to-image diffusion models (T2I-DMs) rely on manipulating text prompts. However, adversarial text prompts are often highly conspicuous—filled with typos, gibberish, or explicit concepts—making them easy to detect or reject by users and safety filters.
With the recent popularity of image-conditioned generation via frameworks like the Image Prompt Adapter (IP-Adapter), the input modality has expanded. This paper addresses a critical security gap: Can image prompts be weaponized to execute stealthy, scalable jailbreak attacks?
The authors introduce the concept of a hijacking attack. Instead of an adversary directly querying a model, the adversary uploads a visually benign but adversarially perturbed image to the web. Unsuspecting users download this image and use it as a prompt for an Image Generation Service (IGS). The hidden perturbation hijacks the model's downstream denoiser, forcing it to output explicit or harmful content, as illustrated in the workflow comparison below:
Methods
The authors propose a method called Attacking Encoder Only (AEO). Because the IP-Adapter relies heavily on features extracted by frozen, open-source upstream image encoders (like CLIP or ArcFace), an attacker does not need white-box access to the entire diffusion pipeline.
The attack optimizes an adversarial image ($x_{adv}$) to match the feature representation of a target NSFW image ($x_{nsfw}$) within the encoder's latent space:
$$\min_{x_{adv}} \text{dist}(f(x_{adv}), f(x_{nsfw})) \quad \text{s.t.} \quad \|x_{adv} - x_b\|_\infty \le \epsilon$$
Where $x_b$ is the original benign image, $f(\cdot)$ is the image encoder, and $\epsilon$ is the perturbation budget.
Key technical aspects include:
- Optimization: Solved using Projected Gradient Descent (PGD) with an $l_\infty$-norm constraint (default $\epsilon = 8/255$).
- Distance Metrics: The authors evaluated both Mean Squared Error (MSE) and Cosine Similarity (COS) to align the latent features.
- Lossless Simulation: To avoid gradient obfuscation, the optimization accounts for standard image formatting (PNG mapping).
Datasets
The method was evaluated across three distinct downstream tasks using standard benchmarks:
- Text-to-Image: 20 benign paintings sampled from the WikiArt dataset, paired with 50 distinct text prompts.
- Image Inpainting (Face Swapping): 20 face images of different identities sampled from CelebA-HQ.
- Virtual Try-on: 20 garment images and 50 human images sampled from the VITON-HD dataset.
Key Insights and Ideas
- The Fidelity-Vulnerability Paradox: A highly accurate IP-Adapter that perfectly captures the nuances of an image prompt is inherently more vulnerable to AEO. As service providers improve model fidelity, they inadvertently make the hijacking attack more effective.
- Direction Over Magnitude: Aligning feature directions using Cosine Similarity yields significantly stronger jailbreaks than minimizing absolute distance via MSE. This is especially true for grid-type IP-Adapters, where cosine similarity in the latent space strongly correlates with semantic similarity in the generated outputs.
- The Failure of Current Defenses: Traditional post-hoc safety checkers and concept-erasing techniques (like ESD-u or SLD) fail because they do not restore the original benign semantics of the image. If a filter blocks the output, the benign user simply experiences a service refusal for a seemingly harmless input, damaging the provider's reputation.
- Adversarial Training as a Shield: Replacing the standard CLIP vision encoder with an adversarially trained counterpart (e.g., using the FARE framework) successfully mitigates the attack by aligning the adversarial examples back to their benign counterparts in the feature space.
Key Results
The authors validated AEO against twelve distinct T2I-IP-DM configurations across text-to-image, inpainting, and virtual try-on tasks (including popular models like Stable Diffusion v1.5, SDXL, and Kolors).
- High Success Rates: In text-to-image tasks, AEO (using Cosine Similarity) boosted the model's Nudity rate to at least 57.3% and the global NSFW rate to at least 77.0% when the adapter weight factor was set to 1.0.
- Virtual Try-on Vulnerability: When attacking the state-of-the-art
IDM-VTONmodel, the attack achieved a 56.2% Nudity rate and an 83.3% NSFW rate, successfully generating explicit content from benign clothing images. - Real-World Transferability: The authors successfully demonstrated the attack against real-world, black-box commercial applications by leveraging surrogate models, showing that using an adversarially trained surrogate (ViT-H-14-FARE) boosted transfer-attack NSFW rates up to 73.6%.
Limitations and Future Work
- Network Channel Assumptions: The optimization assumes a lossless network channel (PNG format). Highly aggressive image compression or resizing applied by some web platforms could degrade the adversarial perturbation and lower attack success rates.
- Encoder Generalization Trade-offs: While robust encoders like FARE successfully defend against the attack, they can slightly degrade the model's generation fidelity on specific domains (e.g., human faces) due to distribution shifts from their underlying training data (ImageNet).
- Advanced Safety Checkers: The paper primarily focuses on bypassing embedding-level prompts; thoroughly evading complex, multi-modal, post-hoc safety checkers via advanced prompt dilution remains an open area for future research.
Summary updated