A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation

Hao Yang, Zhuo Ma, Yang Liu, Yilong Yang, Guancheng Wang, JianFeng MaarXiv 2026 · arXiv

TL;DR

Researchers have developed CrossMPI, a novel prompt injection attack that tricks Large Vision-Language Models (LVLMs) into executing malicious tasks using only subtle, imperceptible noise added to an image. By targeting the model's internal "hidden state" space rather than just the input embeddings, CrossMPI can force a model to ignore a user's text instructions and follow an attacker's intent, achieving a 66.36% success rate in black-box scenarios.

What Problem Does This Paper Address?

Large Vision-Language Models (LVLMs) like GPT-4V or LLaVA combine visual and textual understanding to perform complex tasks. However, they are susceptible to prompt injection, where malicious instructions override the user's original intent.

Existing attacks have a major limitation: they usually only affect one modality. For example, an "image-only" attack might make a model see a cat as a dog, but the model still tries to answer the user's original question. Alternatively, "multimodal" attacks require changing both the text and the image, which is easier for security filters to catch.

The authors of this paper asked: Can we steer the model’s interpretation of a benign text prompt using only image perturbations? This is a "cross-modal" attack, where the visual input fundamentally changes how the model understands the accompanying text.

Figure 1: Comparison of prompt injection paradigms in LVLMs. (A) Image-only attacks perturb the image to change visual perception, while the model still follows the benign text task. (B) Text-image attacks change the task execution by perturbing both the text prompt and the image. (C) Our cross- modal prompt injection perturbs only the image but changes how the model interprets the unchanged text prompt.

Methods

The core innovation of CrossMPI is moving the attack target from the embedding space (where inputs are first turned into numbers) to the hidden state space (where the model actually integrates vision and text). This space is much larger ($10^7$ parameters vs $10^5$), making optimization difficult. To solve this, the authors introduced two key strategies:

  1. Fusion-Critical Layer Selection: Instead of attacking every layer, they use "layer-wise probing" to identify specific layers where the model merges visual evidence with textual intent.
  2. Distance-Decremental Perturbation Budget: Rather than spreading noise uniformly across the image, they use saliency maps (via Grad-ECLIP) to find the "semantic center" of the image. They then allocate more "noise budget" to pixels near this center and less to the background, ensuring the attack is both effective and stealthy.

The final optimization objective (Cross-Modal Perturbation Optimization) jointly optimizes for the target output, aligns internal hidden states with the malicious task, and uses frequency regularization to keep the noise visually imperceptible.

Overview of the CrossMPI attack pipeline, showing layer selection, budget assignment, and the final optimization loop.

Datasets

The authors evaluated CrossMPI across three diverse datasets to prove its generalizability:

  • MSCOCO: A standard image captioning and detection dataset.
  • ImageNet: A massive database of categorized images.
  • TextVQA: A dataset specifically for visual question answering involving text within images.

They used VQA questions as the "benign" user prompts and set target tasks like identifying specific animals or objects (e.g., forcing the model to say "The object is a mobile phone" regardless of the actual image or question).

Key Insights and Ideas

  • The "Middle Layer" Sweet Spot: Traditional adversarial attacks often target the final layers of a network. However, this paper reveals that for LVLMs, the most effective layers for prompt injection are in the middle of the model. This is where the "fusion" of modalities happens; by the time data reaches the final layers, the "decision" has already been made.
  • Hidden States > Embeddings: By attacking the hidden states, the perturbation doesn't just change what the model "sees"—it changes what the model "thinks" the user is asking.
  • Spatial Intelligence: The distance-decremental strategy proves that concentrated, semantically-aware noise is far more effective for cross-model transfer than uniform random noise.
Figure 2: Layer-wise hidden state classification accuracy un- der prompt variants on MiniGPT4-vicuna. The fusion-critical layers are identified as layers 13–15.

Key Results

  • High Success Rate: In black-box settings (where the attacker doesn't know the target model's internals), CrossMPI achieved a 66.36% Attack Success Rate (ASR), outperforming existing baselines by an average of 40.91 percentage points.
  • Strong Transferability: Perturbations optimized on one model (like MiniGPT4) were highly effective when tested on entirely different architectures (like InstructBLIP or BLIVA).
  • Stealthiness: Quantitative metrics (like SSIM and LPIPS) and qualitative visualizations showed that CrossMPI preserves image quality much better than CLIP-based optimization methods, which often leave visible "ghostly" artifacts of the target object.

Limitations and Future Work

The authors note that while CrossMPI is powerful, the generated perturbations are still somewhat dependent on the specific text prompt used during optimization. An attack optimized for "What is in this image?" might not work as well if the user asks "Can you describe this scene?".

Future research directions include developing prompt-agnostic image perturbations that can trigger malicious behavior regardless of the user's specific wording, as well as investigating more robust defense mechanisms to protect LVLMs from these stealthy cross-modal threats.

Summary updated

Abstract

Large vision-language models (LVLMs) have emerged as a powerful paradigm for multimodal intelligence, but their growing deployment also expands the attack surface of prompt injection. Despite this growing concern, existing attacks still suffer from a critical limitation: the injected prompt for one modality only steers the model's interpretation of that singular input. Alternatively, these attacks remain multimodal but fail to achieve cross-modal prompt perturbation. To bridge this gap, we introduce a novel cross-modal prompt injection attack CrossMPI, which can steer the model's interpretation of both textual and visual inputs via image-only prompt injection. Our design is underpinned by the following key breakthroughs. First, we turn the focus of the injected prompt perturbation optimization from the visual embedding space (typically with only $10^5$ parameters) to the model hidden state space (for multimodal information integration and with $10^7$ parameters). Then, two strategies are adopted to mitigate the optimization challenges posed by the larger parameter space. To constrain the optimized model parameter space, we introduce a layer selection strategy that identifies the layers most critical to multimodal integration. Interestingly, deviating from the past experience, our analysis reveals that the optimal layers for LVLM prompt perturbation reside in the middle of the model rather than the last. To constrain the image perturbation space, we propose a new distance-decremental perturbation budget assignment strategy that allocates budgets decrementally as the pixel distance to semantic-critical regions increases. Extensive experiments across multiple LVLMs and datasets show that our method significantly outperforms baseline approaches.
BibTeX
@misc{yang2026cross,
  author = {Hao Yang and Zhuo Ma and Yang Liu and Yilong Yang and Guancheng Wang and JianFeng Ma},
  title = {{A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation}},
  year = {2026},
  eprint = {2605.16090},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.16090},
}

Loading related content...

Figure