Perceptual Threat Sets in Feature Space for Multimodal Jailbreak Training

Vision-language models, which are language models that also read images, lose much of their safety training once an image is attached. A harmful request that the text-only model refuses goes through when it is rendered as text inside a picture, or when a few nearly invisible pixel changes are added. The most efficient fixes so far use adversarial training, meaning they train the model on inputs an attacker has optimized to break it. To keep this affordable, they train only the small projector that turns image features into language-model tokens. These fixes have a blind spot inherited from classic image robustness work. Their attacker searches only inside a small pixel-wise budget, an $L_\infty$ ball, and robustness to that budget is known to transfer poorly to attacks humans perceive as equally small, such as global recoloring, slight warps, or semantic edits. We propose to move the attacker's search from pixels into the image-feature tokens the projector receives. The size of the search region would be set by measuring how image changes that people perceive as small actually move those features.

The goal is a defense that closes more of the multimodal safety gap and costs less to train. If the feature-space region is shaped by many kinds of perceptually small change rather than by pixel noise alone, a defense trained inside it should hold up against attack families it never saw. The inner search would also never run backward through the vision encoder or through expensive functional image transforms. The proposal stands or falls on one test: train against some attack families, evaluate on the families held out.

Core Claim

Our hypothesis is that projector-only adversarial training overfits to the threat model because pixel-$L_\infty$ attacks move the vision encoder's output in a narrow set of directions. Color-function, spatial-warp, typographic, and semantic attacks move it in other directions, and a projector trained on the first set never learns to resist the second. We define the training threat set as an ellipsoid around each image's feature tokens $z$. Its shape comes from the covariance $\Sigma$ of feature displacements $\Delta z$ produced by a large library of random, non-adversarial image changes that a learned perceptual metric rates as small. The inner maximization becomes

$$\max_{\delta:\ \delta^\top \Sigma^{-1} \delta \le \rho^2} \ \mathcal{L}_{\text{jb}}\big(g_\phi(z+\delta),\, x_{\text{text}}\big),$$

where $g_\phi$ is the trainable projector, $\mathcal{L}_{\text{jb}}$ is the loss of producing a harmful target completion (minimized by the attacker as negative log-likelihood), $x_{\text{text}}$ is the paired prompt, and $\rho$ is a budget calibrated so that the ellipsoid covers a fixed fraction of the perceptually small displacements. The outer step trains $g_\phi$ to refuse inside this set while keeping clean behavior. The new part is the threat set: it lives in feature space, and its metric is estimated from perceptual equivalence. A pixel norm does not set it, and neither does an isotropic feature-space ball.

Research Program

The work has three stages. The first is a measurement study. For LLaVA-1.5-7B and Bunny-v1.0-4B, the two models on which projector adversarial training has been reported, we collect feature displacements at the projector input for each attack family. Following Functional Adversarial Attacks, which showed that $L_\infty$-trained classifiers (TRADES) fall to 5.7% accuracy under combined color, flow, and additive attacks, the families are pixel PGD, color-function attacks (ReColorAdv), spatial flow (StAdv), typographic prompts from FigStep and MM-SafetyBench, diffusion-amplified images from HADES, and semantic-factor perturbations from TA-SPA. TA-SPA showed that intent-level features alone transfer to closed models. Comparing the principal subspaces of these displacement sets tests the core hypothesis directly. If pixel-PGD displacements already span the others, the proposal's motivation fails, and we would report that.

The second stage builds $\Sigma$. We sample random draws of each transform family, keep only images under an LPIPS threshold relative to the source, encode them, and fit a low-rank-plus-diagonal covariance. No adversarial optimization is involved. LPIPS is the learned perceptual patch similarity that the functional-attack paper used to compare distortion. Typographic and semantic changes are not "imperceptible," so for those families the ellipsoid stands in for "small change in image content," and we state this caveat explicitly. To prevent leakage, the held-out family is also excluded from the transform library used to fit $\Sigma$.

The third stage is training. We reuse the projector-only recipe and the dynamic loss weighting of ProEAT and E²AT, which trains the projector against PGD image noise and GCG text suffixes. The pixel PGD is replaced by projected gradient steps on $\delta$ inside the ellipsoid, and the GCG text branch is unchanged. The inner loop still backpropagates through the language model, so the savings are bounded. They come from skipping the vision encoder's backward pass and from never rendering functional transforms inside the loop. We will report wall-clock time rather than claim an order-of-magnitude gain.

Baselines

The naive reference point is the undefended model, plus the safety-prompt defense from MM-SafetyBench, which reduced LLaVA-1.5-7B ASR from 77.33% to 15.68% on the Tiny split. Both are run on the same prompts and the same judge.

The strongest existing training-based competitor is ProEAT/E²AT. E²AT reports a weighted ASR on LLaVA of 0.054 on JailBreakV-28K, down from 0.452 with no defense, and ProEAT reports roughly 15 hours of training and a weighted ASR of 0.002 on Bunny-v1.0-4B. We rerun their recipe ourselves with identical harmful-prompt data (AdvBench questions paired with ImageNet images, as in E²AT), the same projector, the same number of outer steps, and the same hyperparameter search budget. We also give them a matched GPU-hour budget.

The strongest training-free competitor is SafeSteer. It steers mid-layer activations along a safety subspace found by SVD, and it reports LLaVA-v1.5 average ASR falling from 62.8% to 5.9% across MM-SafetyBench and JailBreakV-28K, at 2.42 seconds per response. Two more reference points are ESIII from Tit-for-Tat, which reports LLaVA MM-SafetyBench ASR falling from 78.30% to 5.22%, and Contrastive Harmlessness LoRA from HADES, which reports 5.07% on LLaVA-1.5.

We add a multi-threat pixel-space baseline in which ProEAT's inner loop alternates between PGD, ReColorAdv, and StAdv. It is the obvious combination and receives the same compute budget. Two ablations isolate the key ingredient. One is feature-space adversarial training with an isotropic ball of matched volume, which isolates the perceptual metric. The other is training on random perceptual transforms with no inner maximization, which isolates the adversarial search.

Beating the field means two things. On held-out attack families, ASR must be lower than every baseline at matched or lower training cost. Clean MM-Vet and LLaVA-Bench scores must stay within one point of ProEAT's roughly 1% utility loss. Results on the training families are reported, but they do not count as a win.

Why not just…?

Why not just add functional attacks to the pixel inner loop? This is the most serious threat. For the families it trains on, it may match or beat us. Two things could still favor our method. First, the cost: each ReColorAdv or StAdv step carries interpolation and regularizer overhead plus a vision-encoder backward pass. Second, the functional-attack paper itself reports that a model trained against the combined attack still reached only 3.6% accuracy on CIFAR-10. That result suggests stacking pixel-space threats does not buy robustness, although it comes from a small classifier, not a vision-language projector. The leave-one-family-out comparison at matched GPU-hours settles the question. If the multi-threat pixel baseline generalizes as well as we do, our contribution shrinks to a cost saving, and we will say so.

Why not just perturb features inside a plain $L_2$ ball? If the perceptual covariance turns out to be close to isotropic, it adds nothing. The isotropic matched-volume ablation tests this directly, and the measurement study shows beforehand whether displacement subspaces differ across families.

Why not just steer activations at inference, as SafeSteer does? Steering acts on internal states regardless of how the image was perturbed, so it might be threat-model agnostic already. Its source does not test adversaries who optimize to stay orthogonal to the safety subspace, and it adds per-response latency. We run SafeSteer on the same held-out families and under an adaptive attack that penalizes projection onto its subspace. If it holds up there, training-time defenses are not needed for this problem.

Why not just augment with random perceptual transforms? This is the cheapest option, and it is our no-maximization ablation. If it matches our method, the adversarial inner loop is unnecessary.

Proposed Experiments

The main experiment trains on pixel PGD plus GCG text only, then tests on MM-SafetyBench (typography, Stable Diffusion images, and combined), JailBreakV-28K, the HADES benchmark, and VAJM-style semantic perturbations generated with TA-SPA. ASR is judged by the same LLM judge across all methods. We then rotate which families are held out. We found no reported result on Lacuna for functional color or flow attacks against any vision-language jailbreak defense, so we establish those numbers ourselves for every baseline.

Robustness to non-safety attacks is measured with COCO captioning CIDEr and OK-VQA accuracy under $\epsilon=1/255$ APGD and under ReColorAdv. This follows On the Adversarial Robustness of Multi-Modal Foundation Models, which reports OpenFlamingo CIDEr falling from 84.01 to 9.59 under that APGD attack. Utility is measured with MM-Vet and LLaVA-Bench.

The decisive check is a fully adaptive white-box attack. The attacker knows the trained projector and optimizes in pixel, color-function, and typographic spaces end to end. If our held-out-family advantage disappears under this attack, the feature-space threat set has only moved the blind spot elsewhere. Training wall-clock and peak memory are reported against the 15-hour ProEAT figure under our own reruns.