TL;DR
Unified Multimodal Models (UMMs) that both generate and understand text and images introduce a dangerous new attack surface: Cross-Modal Generative Injection (CMGI). By tricking a model into generating visual scenes of a harmful event's "setup" and "aftermath," the proposed STaR-Attack forces the model to infer and answer an unmodified malicious query in a guessing game—achieving up to a 93% attack success rate without semantic drift.
What Problem Does This Paper Address?
Emerging Unified Multimodal Models (UMMs), such as Janus-Pro, BAGEL, and Gemini Flash, integrate multimodal understanding and image generation into a single system. While powerful, this tight coupling creates an overlooked security vulnerability: an attacker can use the model’s generative capacity to craft context-heavy images, and then exploit its understanding capacity to interpret that context to bypass safety filters.
Existing jailbreak attacks suffer from two major flaws in this setting:
- They are designed for single modalities (text-only LLMs or vision-language MLLMs) and fail to exploit cross-modal generation–understanding dynamics.
- They rely heavily on prompt rewriting, which leads to severe semantic drift—meaning the rewritten prompt might technically trigger an unfiltered response, but the response no longer answers the attacker's original harmful question.
Methods
The authors propose STaR-Attack (Spatio-Temporal and Narrative Reasoning Attack), a multi-turn framework structured around a classic three-act narrative:
[Act 1: Setup Scene] --> Target UMM generates pre-event image
[Act 2: Hidden Climax] --> The harmful action (never explicitly mentioned)
[Act 3: Resolution Scene] --> Target UMM generates post-event image
│
▼
[Final Turn: Guessing Game] --> Target selects & answers the original query from candidates
- Narrative Crafting via Hidden Climax: Instead of asking the model to execute a harmful task directly, an external uncensored LLM creates descriptions for a pre-event setup ($S_{pre}$) and a post-event resolution ($S_{post}$). In the first two dialogue turns, the target UMM generates images for both scenes.
- "Guess and Answer" Game: In the final turn, the attacker presents an image-based riddle asking the model to deduce the user's intent from the generated pictures. The original, unmodified harmful query $Q$ is embedded alongside several benign, semantically distant distractor questions. Because $Q$ is presented as-is, semantic drift is completely avoided.
- Dynamic Difficulty Mechanism: If the model refuses to answer, the attack escalates through four difficulty levels ($D_0$ to $D_3$), increasing the number of benign distractor queries (from 0 up to 7). Counter-intuitively, adding more benign options increases context complexity and masks malicious intent, compelling the model to lean more heavily on the visual narrative history and comply.
Datasets
The authors evaluated STaR-Attack using two standard red-teaming benchmarks:
- AdvBench: 520 harmful instructions covering cybercrime, dangerous advice, misinformation, and threats.
- HarmBench: 400 textual behavior prompts across standard, copyright, and contextual functional types.
Evaluations were performed on open-source UMMs (Janus-7B-Pro, BAGEL-7B-MoT) and closed-source commercial APIs (Gemini-2.0-Flash, Gemini-2.5-Flash).
Key Insights and Ideas
- UMMs Weaponize Their Own Generation: The attacker never uploads an adversarial image. Instead, the model creates the visual context itself from benign-sounding scene descriptions, effectively bypassing input-level safety filters.
- Preserving Harmfulness via "Relevance": Standard Attack Success Rate (ASR) is often misleading because baseline attacks dilute the original query's meaning. The authors highlight Relevant ASR (RASR)—requiring the response to be both unsafe and genuinely answer the original query.
- Hiding in Plain Sight: Increasing the distractor questions in the guessing game does not confuse the model; rather, it cloaks the harmful query within innocuous context while pushing the model’s reasoning engine to rely strictly on the prior narrative turns.
Key Results
STaR-Attack consistently outperformed state-of-the-art single-turn (PAIR, ReNeLLM, FlipAttack) and multi-turn (X-Teaming) baselines across all models and metrics:
| Model | AdvBench ASR | AdvBench RASR | HarmBench ASR | HarmBench RASR |
|---|---|---|---|---|
| Janus-Pro | 93.06% | 71.87% | 92.75% | 45.00% |
| (Best Baseline) | 76.15% (PAIR) | 13.27% (PAIR) | 76.75% (X-Team) | 19.50% (PAIR) |
| BAGEL | 89.60% | 57.23% | 89.00% | 46.25% |
| (Best Baseline) | 87.13% (PAIR) | 20.00% (ReNeLLM) | 86.75% (PAIR) | 40.00% (Text) |
| Gemini-2.0-Flash | 93.06% | 65.32% | 90.75% | 61.50% |
| (Best Baseline) | 86.73% (FlipAttack) | 45.58% (FlipAttack) | 86.50% (FlipAttack) | 58.50% (FlipAttack) |
| Gemini-2.5-Flash | 88.05% | 45.47% | 88.50% | 52.00% |
| (Best Baseline) | 81.15% (FlipAttack) | 29.62% (FlipAttack) | 83.00% (FlipAttack) | 52.50% (FlipAttack) |
Crucially, while baselines like ReNeLLM achieved high ASR on AdvBench (86.54%), their RASR plummeted to 20.0%, proving that their jailbreaks suffered heavy semantic drift. STaR-Attack maintained high cosine similarity (~0.80–0.95) to the original questions.
Limitations and Future Work
- Requires Multi-Turn Architecture: STaR-Attack relies on multi-turn dialogue state; unified models that only operate in single-turn mode (such as BLIP3-o or Show-o2) cannot be evaluated under this specific protocol.
- Dependency on Prompt Templates: Model vulnerabilities vary with internal conversation templating. For example, Janus-Pro (which uses explicit role annotations) was much more sensitive to multi-turn interactions than BAGEL (which flattens turns into token streams).
- Defensive Implications: Current safety alignment treats text comprehension and image generation in isolation. The authors urge the research community to develop cross-modal safety alignment that monitors context built across mixed-modal, multi-turn interactions.
Summary updated