TL;DR
This paper provides the first unified, comprehensive survey of adversarial attacks and defenses for diffusion models across image, video, and 3D modalities. By introducing a formalized mathematical framework and a task-centric taxonomy, the authors map out the current arms race between proactive "content immunization" and adaptive countermeasures.
What Problem Does This Paper Address?
Diffusion models (DMs) have rapidly become the dominant framework for visual generation, powering state-of-the-art tools for image synthesis, video animation, and 3D asset creation. However, their widespread public availability has opened the door to massive misuse, including identity theft via unauthorized personalization, deepfakes, copyright infringement, and malicious editing.
Traditional countermeasures like deepfake detection or watermarking are reactive and easily bypassed. Consequently, the research community has shifted toward proactive content immunization—adding imperceptible adversarial perturbations ("cloaks") to clean data before it is published, ensuring that any downstream attempt by a diffusion model to manipulate or train on the data results in a corrupted, unusable output.
Prior survey papers in this domain have been strictly image-centric or focused narrowly on deepfake detection. This paper addresses this fragmentation by providing a unified review that spans multiple visual modalities (image, video, and 3D) under a single adversarial framework.
Methods
The authors formalize the adversarial landscape using a unified three-actor game: Alice (the content owner), Eve (the adversary attempting unauthorized generation), and Bob (the defender applying an imperceptible cloak $\delta$).
Rather than categorizing methods by their perturbation space, the authors introduce a task-centric taxonomy. They organize the literature first by modality, then by attack vs. defense, and finally group them chronologically by the specific generative task they target:
- Image Tasks: Personalization (e.g., disrupting DreamBooth/LoRA), Style/Copyright protection (e.g., preventing artistic style mimicry), Editing/Inpainting, and Face-swapping.
- Video Tasks: Image-to-Video (I2V) animation, Talking-head generation, Video-to-Video (V2V) editing, and Text-to-Video (T2V) safety.
- 3D Tasks: Editing 3D Gaussian Splatting (3DGS) assets, viewpoint poisoning, and computational cost attacks.
Datasets
As a comprehensive survey, this paper does not introduce a new dataset but consolidates the evaluation sources used across the literature (detailed in the paper's Table 6):
- Image: Facial datasets (CelebA-HQ, VGGFace2, FFHQ), art corpora (WikiArt, ArtBench-10), general vision datasets (ImageNet, COCO), and dedicated editing benchmarks (MagicBrush, EditBench).
- Video: Action and segmentation datasets (DAVIS 2017, UCF101), human animation corpora (TikTok, CelebV-Text, HDTF), and safety benchmarks (T2VSafetyBench, VIOLENT).
- 3D: Synthetic reconstruction sources (NeRF-Synthetic, Google Scanned Objects), real captures (LLFF, Mip-NeRF 360, Tanks & Temples), and 3D editing testbeds (IN2N, NeRF-Art).
Key Insights and Ideas
- The U-Net to DiT Shift: The architectural evolution from convolutional U-Nets to Diffusion Transformers (DiTs) has broken many existing image cloaks. U-Net cloaks typically target localized cross-attention maps, whereas DiTs use global self-attention that scatters a localized perturbation's impact, requiring entirely new token-aware adversarial designs.
- Modality-Specific Bottlenecks: Extending protections beyond static images introduces severe structural constraints. Video cloaks must survive aggressive compression codecs and temporal smoothing, while 3D cloaks must be "lifted" directly into 3D representation parameters (like Gaussian positions or opacities) to maintain multi-view consistency.
- The Purification Arms Race: The party deploying the generative model often acts as an adaptive adversary. They can easily neutralize fixed perturbations using preprocessing techniques like lossy JPEG compression, blurring, or diffusion-based purification (e.g., DiffPure). To survive, modern cloaks must be optimized using minimax formulations that explicitly anticipate these purifiers.
Key Results
The survey provides an in-depth analysis of evaluation settings, consolidating metrics into six recurring categories:
- Effectiveness: Measures how well the generation degrades using metrics like FID, LPIPS, SSIM, and CLIP similarity.
- Imperceptibility: Ensures the cloak remains invisible to humans using PSNR, MS-SSIM, or VLM/human studies.
- Efficiency: Highlights a massive computational shift. Early methods relied on expensive per-sample optimization, whereas recent frameworks leverage pre-trained feed-forward generators (e.g., FastProtect, DiffVax) to achieve 200× to 250,000× speedups, making real-time immunization practical.
Limitations and Future Work
The authors identify three critical open challenges for the field:
- Fragility Against Unseen Purifiers: Most current cloaks fail when faced with novel preprocessing pipelines. The field needs to transition from empirical trial-and-error toward certified defenses that provide mathematical guarantees against broad classes of purifiers.
- Black-Box Generalization: Immunizers typically require white-box access to the target model's gradients. They transfer poorly to real-world commercial APIs (like Midjourney or Sora) where weights, architectures, and preprocessing steps are hidden.
- Standardization in Advanced Modalities: While image-based immunization is relatively mature, video and 3D lack standardized benchmarks, unified threat models, and robust evaluation suites to reliably measure progress.
Summary updated