Abstract
Adversarial attacks can mislead neural network classifiers. The defense against adversarial attacks is important for AI safety. Adversarial purification is a family of approaches that defend adversarial attacks with suitable pre-processing. Diffusion models have been shown to be effective for adversarial purification. Despite their success, many aspects of diffusion purification still remain unexplored. In this paper, we investigate and improve upon three limiting designs of diffusion purification: the use of an improved diffusion model, advanced numerical simulation techniques, and optimal control of randomness. Based on our findings, we propose Purify++, a new diffusion purification algorithm that is now the state-of-the-art purification method against several adversarial attacks. Our work presents a systematic exploration of the limits of diffusion purification methods.
Introduction
Deep learning models are notoriously fragile. "Adversarial attacks"—tiny, human-invisible perturbations added to an image—can easily trick a high-performing classifier into misclassification. To combat this, researchers use adversarial purification. Instead of retraining the classifier, which is computationally expensive and often specific to one type of attack, purification acts as a pre-processor. It takes a potentially malicious image, adds noise to "drown out" the attack, and then uses a generative model to reconstruct the original clean image. While Diffusion Models have shown promise, previous methods like DiffPure often relied on design choices optimized for image generation rather than defense.
Methodology
Purify++ improves upon three specific design pillars of diffusion-based purification:
- Variance Exploding (VE) Models: Most diffusion models use "Variance Preserving" (VP) schedules. The authors demonstrate that VP models cause different images to become indistinguishable too quickly (quadratic interaction time). By switching to Variance Exploding (VE) models (specifically the EDM framework), which have a linear interaction time, the model retains more useful information during the purification process.
- Efficient Simulation: The framework moves away from simple first-order solvers to Heun’s second-order method. While high-order solvers are critical for generating images from scratch, purification starts with a "slightly noisy" image. This allows for a significant reduction in diffusion steps—as few as 20—without substantial loss in protection.
- Optimal Control of Randomness: Purification can be deterministic (ODE) or stochastic (SDE). Purify++ introduces a mixing coefficient ($\lambda$) that blends these paths. A "sweet spot" of randomness ($\lambda \approx 0.75$) was found to provide superior defense compared to purely deterministic or purely stochastic approaches.
Theoretical Insights: Interaction Time
A key theoretical contribution is the analysis of "Interaction Time." In standard generative AI, fast convergence is desirable. However, for purification, if the forward process merges the "adversarial" and "clean" distributions too quickly, the model loses the ability to reconstruct the original signal. VE models are superior because they keep these distributions distinct for longer periods, as shown in the robust accuracy comparisons.
Experimental Results
The researchers evaluated Purify++ on standard benchmarks including MNIST and CIFAR-10 against a variety of attacks (Square Attack, SPSA, PGD, and AutoAttack).
- State-of-the-Art Robustness: On CIFAR-10, Purify++ achieved 91.21% robust accuracy against $\ell_\infty$ Square Attacks, outperforming previous methods like DiffPure (85.42%).
- Adaptive Attack Resilience: Against BPDA+EOT attacks designed to bypass pre-processors, Purify++ maintained 83.33% accuracy.
- Efficiency: The model is remarkably stable; reducing diffusion steps from 200 to 25 resulted in only a ~1.3% drop in robust accuracy, enhancing its viability for real-world deployment.
Conclusion
Purify++ demonstrates that tailoring diffusion models specifically for defense—rather than simply adopting generation-focused defaults—leads to significant gains in adversarial robustness. While computational overhead remains a challenge for real-time applications, the systematic exploration of VE models and controlled randomness sets a new benchmark for adversarial purification. Future work will involve scaling these methods to higher-resolution datasets like ImageNet and investigating further evolutions in adaptive attacks.
Summary updated