A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalities

Ozgur Kara, Tarik Can Ozden, Furkan Horoz, Zeqian Long, Haotian Xue, Yipu Chen, Oguzhan Akcin, Yongxin Chen, James Matthew RehgarXiv 2026 · arXiv

TL;DR

This paper provides the first unified, comprehensive survey of adversarial attacks and defenses for diffusion models across image, video, and 3D modalities. By introducing a formalized mathematical framework and a task-centric taxonomy, the authors map out the current arms race between proactive "content immunization" and adaptive countermeasures.

What Problem Does This Paper Address?

Diffusion models (DMs) have rapidly become the dominant framework for visual generation, powering state-of-the-art tools for image synthesis, video animation, and 3D asset creation. However, their widespread public availability has opened the door to massive misuse, including identity theft via unauthorized personalization, deepfakes, copyright infringement, and malicious editing.

Traditional countermeasures like deepfake detection or watermarking are reactive and easily bypassed. Consequently, the research community has shifted toward proactive content immunization—adding imperceptible adversarial perturbations ("cloaks") to clean data before it is published, ensuring that any downstream attempt by a diffusion model to manipulate or train on the data results in a corrupted, unusable output.

Prior survey papers in this domain have been strictly image-centric or focused narrowly on deepfake detection. This paper addresses this fragmentation by providing a unified review that spans multiple visual modalities (image, video, and 3D) under a single adversarial framework.

Table 1: Comparison with existing surveys. ✓marks a modality that is a primary subject, ✗peripheral or absent; venue years give the first public release. Ours is the only survey covering image, video, and 3D together.

Methods

The authors formalize the adversarial landscape using a unified three-actor game: Alice (the content owner), Eve (the adversary attempting unauthorized generation), and Bob (the defender applying an imperceptible cloak $\delta$).

Figure 2: Diffusion-model immunization across modalities. Each row represents an Alice–Eve–Bob sce- nario: Alice publishes content x (col. 1); Bob applies an imperceptible cloak δ under budget ∥δ∥X ≤ϵ (col. 2); Eve executes her generative pipeline Dθ(z, x ⊕δ, c) on the protected asset (col. 3), yielding a degraded or corrupted output (col. 4).

Rather than categorizing methods by their perturbation space, the authors introduce a task-centric taxonomy. They organize the literature first by modality, then by attack vs. defense, and finally group them chronologically by the specific generative task they target:

  • Image Tasks: Personalization (e.g., disrupting DreamBooth/LoRA), Style/Copyright protection (e.g., preventing artistic style mimicry), Editing/Inpainting, and Face-swapping.
  • Video Tasks: Image-to-Video (I2V) animation, Talking-head generation, Video-to-Video (V2V) editing, and Text-to-Video (T2V) safety.
  • 3D Tasks: Editing 3D Gaussian Splatting (3DGS) assets, viewpoint poisoning, and computational cost attacks.
Figure 3: Task-centric taxonomy of adversarial attacks and defenses for diffusion models across (a) im- age, (b) video, and (c) 3D. Methods are grouped by generative task within each modality; image and 3D defenses are collected in separate branches, while video counter-defenses remain nascent and are discussed in the text (§3.2). Red marks attacks and green marks defenses.

Datasets

As a comprehensive survey, this paper does not introduce a new dataset but consolidates the evaluation sources used across the literature (detailed in the paper's Table 6):

  • Image: Facial datasets (CelebA-HQ, VGGFace2, FFHQ), art corpora (WikiArt, ArtBench-10), general vision datasets (ImageNet, COCO), and dedicated editing benchmarks (MagicBrush, EditBench).
  • Video: Action and segmentation datasets (DAVIS 2017, UCF101), human animation corpora (TikTok, CelebV-Text, HDTF), and safety benchmarks (T2VSafetyBench, VIOLENT).
  • 3D: Synthetic reconstruction sources (NeRF-Synthetic, Google Scanned Objects), real captures (LLFF, Mip-NeRF 360, Tanks & Temples), and 3D editing testbeds (IN2N, NeRF-Art).

Key Insights and Ideas

  • The U-Net to DiT Shift: The architectural evolution from convolutional U-Nets to Diffusion Transformers (DiTs) has broken many existing image cloaks. U-Net cloaks typically target localized cross-attention maps, whereas DiTs use global self-attention that scatters a localized perturbation's impact, requiring entirely new token-aware adversarial designs.
  • Modality-Specific Bottlenecks: Extending protections beyond static images introduces severe structural constraints. Video cloaks must survive aggressive compression codecs and temporal smoothing, while 3D cloaks must be "lifted" directly into 3D representation parameters (like Gaussian positions or opacities) to maintain multi-view consistency.
  • The Purification Arms Race: The party deploying the generative model often acts as an adaptive adversary. They can easily neutralize fixed perturbations using preprocessing techniques like lossy JPEG compression, blurring, or diffusion-based purification (e.g., DiffPure). To survive, modern cloaks must be optimized using minimax formulations that explicitly anticipate these purifiers.

Key Results

The survey provides an in-depth analysis of evaluation settings, consolidating metrics into six recurring categories:

  1. Effectiveness: Measures how well the generation degrades using metrics like FID, LPIPS, SSIM, and CLIP similarity.
  2. Imperceptibility: Ensures the cloak remains invisible to humans using PSNR, MS-SSIM, or VLM/human studies.
  3. Efficiency: Highlights a massive computational shift. Early methods relied on expensive per-sample optimization, whereas recent frameworks leverage pre-trained feed-forward generators (e.g., FastProtect, DiffVax) to achieve 200× to 250,000× speedups, making real-time immunization practical.

Limitations and Future Work

The authors identify three critical open challenges for the field:

  • Fragility Against Unseen Purifiers: Most current cloaks fail when faced with novel preprocessing pipelines. The field needs to transition from empirical trial-and-error toward certified defenses that provide mathematical guarantees against broad classes of purifiers.
  • Black-Box Generalization: Immunizers typically require white-box access to the target model's gradients. They transfer poorly to real-world commercial APIs (like Midjourney or Sora) where weights, architectures, and preprocessing steps are hidden.
  • Standardization in Advanced Modalities: While image-based immunization is relatively mature, video and 3D lack standardized benchmarks, unified threat models, and robust evaluation suites to reliably measure progress.

Summary updated

Abstract

Diffusion models have become the dominant family of generative models in the visual domain. However, their widespread public availability enables misuse at scale, motivating a rapidly growing body of research on adversarial attacks and defenses. This survey provides, to our knowledge, the first unified review of this literature across three visual modalities: image, video, and 3D. We introduce a comprehensive, task-centric taxonomy: we first divide the literature by modality; within each modality, we separate methods into attacks and defenses, and then group them by the generative task they target, presenting them chronologically within each task. Moreover, we provide an in-depth analysis of their evaluation settings, consolidating the datasets, metrics, and benchmarks used to assess them. We conclude by identifying several open challenges and outlining concrete future research directions. Project Webpage: https://github.com/ozgurkara99/awesome-adv-attack-defense-on-diffusion
BibTeX
@misc{kara2026survey,
  author = {Ozgur Kara and Tarik Can Ozden and Furkan Horoz and Zeqian Long and Haotian Xue and Yipu Chen and Oguzhan Akcin and Yongxin Chen and James Matthew Rehg},
  title = {{A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalities}},
  year = {2026},
  eprint = {2609.05503},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.05503},
}
Figure