Ceci n'est pas une pomme: Adversarial Illusions in Multi-Modal Embeddings

Eugene Bagdasaryan, Rishi Dev Jha, Tingwei Zhang, Vitaly ShmatikovSubmission to ICLR 2024 · Rejected

TL;DR

Multi-modal models like ImageBind map images, text, and audio into a shared representation space, but their "organic" cross-modal alignment is surprisingly loose. The authors introduce adversarial illusions, showing that imperceptible perturbations to an input in one modality (e.g., an image or audio clip) can force its embedding to collide with an arbitrary target in another modality (e.g., text). Because downstream systems blindly trust this shared space, this single attack misleads classification, image generation, and text generation simultaneously without needing access to the downstream models.

What Problem Does This Paper Address?

Modern multi-modal embedding architectures (most notably ImageBind) encode disparate data types—such as text, images, and audio—into a single, joint embedding space. Downstream applications (such as diffusion models, instruction-following LLMs like PandaGPT, and zero-shot classifiers) rely on this shared space under the assumption that proximity in the embedding space guarantees semantic equivalence across modalities.

Prior adversarial attacks on multi-modal pipelines either targeted a single modality (e.g., text-only collisions), relied on large visible perturbations, or focused on downstream task-specific vulnerabilities (such as jailbreaks). This paper asks a deeper question: Can an attacker create imperceptible, downstream-agnostic perturbations in one modality that hijack the representation of an arbitrary target in a completely different modality?

Methods

The authors introduce cross-modal illusions, a white-box attack against multi-modal encoders that is entirely agnostic to downstream tasks:

  • Objective Formulation: Given an input $x^{m_1}$ in modality $m_1$ and an adversary-chosen target $a^{m_2}$ in modality $m_2$, the attack optimizes an adversarial perturbation $\delta$ to maximize cosine similarity in embedding space: $$\ell = 1 - \cos(\theta^{m_1}(x^{m_1} + \delta), \theta^{m_2}(a^{m_2}))$$ where $\theta^m$ represents the modality-specific encoder.

  • Optimization: Perturbations are computed using the Iterative Fast Gradient Sign Method (I-FGSM) under an $L_\infty$ constraint:

  • For images, perturbations are bounded by $\epsilon_V / 255$ with $\epsilon_V \in \{1, 4, 8, 16, 32\}$.
  • For audio, inputs are converted to Mel spectrograms (treating decibel levels analogously to pixel intensities), bounded by $\epsilon_A \in \{0.005, \dots, 0.5\}$.
  • Zero Downstream Knowledge: The adversary requires white-box access to the embedding encoders ($\theta$), but has zero access to or knowledge of the downstream models (e.g., unCLIP, PandaGPT).

Datasets

The authors evaluated adversarial illusions on 100-sample subsets using permuted bipartite pairings (source inputs paired with random, unrelated target texts):

  • ImageNet: Used to test natural (image-to-text) alignment, zero-shot classification, and image generation. Labels were formatted into standard text prompts (e.g., "A photo of a {label}.").
  • AudioCaps: A dataset of audio clips and text captions used to test emergent (audio-to-text) alignment on retrieval tasks, since ImageBind was not directly trained on audio-text pairs.

Key Insights and Ideas

  • Organic alignment is remarkably weak: The baseline cosine similarity between semantically matching pairs in ImageBind is low—roughly 0.29 for ImageNet (image-text) and 0.21 for AudioCaps (audio-text).
  • Adversarial alignment easily dominates organic alignment: Even microscopic perturbations allow an unrelated input to align closer to a target text than real, matching data does. At just $\epsilon_V = 1/255$, adversarial image-to-text alignment hits 0.5741 (nearly double organic alignment).
  • Universal downstream failure: Because multi-modal downstream tools treat the latent space as pure semantics, poisoning the embedding fundamentally corrupts every downstream task at once:
  • An image of René Magritte’s apple painting perturbed to match the text "Everything we see hides another thing..." causes image generators to output a collage of faces and prompt LLMs to describe faces instead of an apple.
  • An audio recording of a howling wolf perturbed toward the image of a sheep causes classifiers to output "ram, tup (sheep)" and generative models to generate sheep pictures.

Key Results

  • Zero-Shot Classification (ImageNet):
  • At a tiny perturbation bound of $\epsilon_V = 1/255$, target Top-1 accuracy reaches 93%.
  • At standard bounds ($\epsilon_V \ge 4/255$), target Top-1 accuracy reaches 100%, with cosine similarity reaching 0.9554 at $\epsilon_V = 16/255$.
  • Conditional Image Generation (PandaGPT / unCLIP):
  • Images generated from perturbed ImageNet embeddings matched the target label with 64% Top-1 and 92% Top-5 accuracy (evaluated via a ViT-B/16 classifier). This outperformed the generation accuracy of unperturbed baseline images.
  • Audio Retrieval (AudioCaps):
  • While normal audio-caption retrieval achieves 12% Top-1 accuracy, audio illusions achieved 99% Top-1 retrieval of the adversary’s target caption at $\epsilon_A = 0.05$.

Limitations and Future Work

  • White-box Assumption: The attack relies on full white-box access to the embedding model encoders. Devising black-box attacks is non-trivial when only downstream generative outputs (like noisy diffusion outputs) are observable.
  • Single Model Target: Experiments focused on ImageBind, as it was the only fully open-source, multi-modal unified embedding model available at the time.
  • Downstream Imperfections: Current downstream generative architectures (such as BindDiffusion or PandaGPT) sometimes misinterpret cross-modal embeddings (e.g., PandaGPT occasionally describes audio as visual imagery) independently of adversarial attacks.
  • Defenses are Hard: Standard adversarial training in contrastive multi-modal settings severely degrades utility on clean data due to data sparsity and modality gaps. The authors suggest that future defenses must look into high-dimensional anomaly detection or enforcing strict per-modality geometric constraints.

Summary updated

Abstract

Multi-modal embeddings encode images, sounds, texts, videos, etc. into a single embedding space, aligning representations across modalities (e.g., associate an image of a dog with a barking sound). We show that multi-modal embeddings can be vulnerable to an attack we call ``adversarial illusions.'' Given an image or a sound, an adversary can perturb it so as to make its embedding close to an arbitrary, adversary-chosen input in another modality. This enables the adversary to align any image and any sound with any text. Adversarial illusions exploit proximity in the embedding space and are thus agnostic to downstream tasks. Using ImageBind embeddings, we demonstrate how adversarially aligned inputs, generated without knowledge of specific downstream tasks, mislead image generation, text generation, and zero-shot classification.
BibTeX
@misc{bagdasaryan2023ceci,
  author = {Eugene Bagdasaryan and Rishi Dev Jha and Tingwei Zhang and Vitaly Shmatikov},
  title = {{Ceci n'est pas une pomme: Adversarial Illusions in Multi-Modal Embeddings}},
  year = {2023},
  eprint = {2308.11804},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.11804},
}

Loading related content...

Figure