Diffusion Models for Adversarial Purification

Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, Anima Anandkumar

Abstract

Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model. These methods do not make assumptions on the form of attack and the classification model, and thus can defend pre-existing classifiers against unseen threats. However, their performance currently falls behind adversarial training methods. In this work, we propose DiffPure that uses diffusion models for adversarial purification: Given an adversarial example, we first diffuse it with a small amount of noise following a forward diffusion process, and then recover the clean image through a reverse generative process. To evaluate our method against strong adaptive attacks in an efficient and scalable way, we propose to use the adjoint method to compute full gradients of the reverse generative process. Extensive experiments on three image datasets including CIFAR-10, ImageNet and CelebA-HQ with three classifier architectures including ResNet, WideResNet and ViT demonstrate that our method achieves the state-of-the-art results, outperforming current adversarial training and adversarial purification methods, often by a large margin.

Introduction

Deep neural networks are notoriously fragile; tiny, imperceptible "adversarial perturbations" can trick them into misclassifying images with high confidence. The standard defense is adversarial training (AT), where models are trained directly on these perturbed examples. However, AT has major drawbacks: it is computationally expensive, often degrades performance on clean images, and typically only defends against the specific types of attacks encountered during training.

An alternative is adversarial purification, which uses a generative model to "wash away" perturbations before classification. Historically, purification methods using GANs or Energy-Based Models (EBMs) have failed to match the performance of AT, largely due to poor generative quality or "gradient masking"—a phenomenon where the defense appears robust only because the attacker cannot calculate an effective gradient to bypass it. DiffPure addresses these limitations by leveraging the superior generative capabilities of diffusion models.

The DiffPure Framework

DiffPure utilizes continuous-time diffusion models to map adversarial examples back to the clean data manifold. The process consists of two primary stages:

  1. Forward Diffusion: Given a potentially adversarial image, the method adds a controlled amount of Gaussian noise following a forward Stochastic Differential Equation (SDE). This step is designed to "wash out" the local adversarial structures that trick the classifier.
  2. Reverse Denoising: The method then solves a reverse-time SDE to reconstruct a clean image from the noisy version. Because diffusion models are trained to capture the true underlying data distribution, the reconstructed image is pushed back toward the manifold of natural, clean images while preserving the original semantic content.

Evaluating Against Adaptive Attacks

A critical technical contribution of this work is the use of the adjoint method for evaluation. To rigorously test a defense, one must assume an "adaptive attacker" who possesses full knowledge of the purification process. Calculating gradients through an SDE solver is traditionally memory-intensive. The adjoint method allows the authors to compute full gradients with constant memory cost, ensuring that the reported robustness is genuine and not a byproduct of numerical instability or gradient masking.

Key Insights

The Diffusion "Sweet Spot" ($t^*$)

There is a fundamental trade-off in the purification process. If too little noise is added during the forward pass, the adversarial perturbation may survive. If too much noise is added, the image's original identity may be destroyed (e.g., a "cat" might be reconstructed as a "dog"). DiffPure identifies an optimal diffusion timestep $t^*$ that balances perturbation removal with identity preservation.

Plug-and-Play Robustness

Unlike adversarial training, DiffPure does not require re-training the classifier. It acts as a modular "shield" that can be placed in front of any pre-trained, high-accuracy model. This makes it particularly valuable for Vision Transformers (ViTs), which are notoriously difficult and expensive to train using standard adversarial methods.

Generalization to Unseen Threats

Because the purification process is independent of the specific attack type, it performs exceptionally well against "unseen" attacks, such as spatial transformations, where traditional adversarial training often fails to generalize.

Experimental Results

The authors evaluated DiffPure across CIFAR-10, ImageNet, and CelebA-HQ using various architectures, including ResNet, WideResNet, and ViT.

  • State-of-the-Art Robustness: On CIFAR-10, DiffPure achieved a robust accuracy of 70.64% against AutoAttack ($\ell_\infty$), an absolute improvement of +5.44% over the best adversarial training baseline without extra data.
  • ImageNet Scalability: On ImageNet, it improved robust accuracy by up to +7.68% over previous benchmarks.
  • Unseen Attacks: Against spatially transformed attacks (StAdv), DiffPure outperformed standard adversarial training by a staggering 36% in robust accuracy.
  • Generative Superiority: On CelebA-HQ, DiffPure achieved approximately 90% robust accuracy, while GAN-based purification methods struggled to exceed 75%, highlighting the advantage of diffusion over previous generative architectures.

Limitations and Future Work

The primary limitation of DiffPure is computational latency. Solving an SDE for every image during inference is significantly slower than a standard forward pass. This currently limits the application of DiffPure in real-time environments like autonomous driving. Future research may focus on accelerating diffusion sampling through specialized solvers or optimizing diffusion models specifically for the purification task rather than general image synthesis.

Summary updated

Abstract

Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model. These methods do not make assumptions on the form of attack and the classification model, and thus can defend pre-existing classifiers against unseen threats. However, their performance currently falls behind adversarial training methods. In this work, we propose DiffPure that uses diffusion models for adversarial purification: Given an adversarial example, we first diffuse it with a small amount of noise following a forward diffusion process, and then recover the clean image through a reverse generative process. To evaluate our method against strong adaptive attacks in an efficient and scalable way, we propose to use the adjoint method to compute full gradients of the reverse generative process. Extensive experiments on three image datasets including CIFAR-10, ImageNet and CelebA-HQ with three classifier architectures including ResNet, WideResNet and ViT demonstrate that our method achieves the state-of-the-art results, outperforming current adversarial training and adversarial purification methods, often by a large margin. Project page: https://diffpure.github.io.
BibTeX
@inproceedings{nie2022diffusion,
  author = {Weili Nie and Brandon Guo and Yujia Huang and Chaowei Xiao and Arash Vahdat and Anima Anandkumar},
  title = {{Diffusion Models for Adversarial Purification}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=5pOwzFzLhkx},
}
Figure