VideoPure: Diffusion-Based Adversarial Purification for Video Recognition

Kaixun Jiang, Zhaoyu Chen, Jiyuan Fu, Lingyi Hong, J Li, Wenqiang Zhang

Introduction

Deep Neural Networks (DNNs) have achieved remarkable success in video recognition, yet they remain notoriously vulnerable to adversarial attacks. These attacks involve small, often imperceptible perturbations to input data that can mislead models into making incorrect predictions with high confidence. While adversarial defense has been extensively studied in the image domain, protecting video recognition models presents a significantly greater challenge due to the added temporal dimension and the resulting massive attack surface.

Traditional defenses, such as adversarial training, require retraining models on poisoned datasets, which is computationally expensive and often fails to generalize to novel attack types. Adversarial purification offers a more flexible alternative by "cleaning" inputs before they reach the model. However, existing purification methods—particularly those leveraging diffusion models—are typically designed for static images. When applied to video, these methods often treat frames independently, destroying temporal consistency and leading to poor defense performance and high latency.

To address these limitations, we present VideoPure, the first diffusion-based adversarial purification framework specifically engineered for video. VideoPure is a "plug-and-play" module that requires no retraining of the underlying recognition model and effectively strips away malicious perturbations while preserving the video's structural integrity.

Methodology

VideoPure operates through a three-stage process designed to balance noise removal with temporal coherence.

1. Temporal DDIM Inversion

Standard diffusion models add random Gaussian noise to inputs. In contrast, VideoPure utilizes Denoising Diffusion Implicit Models (DDIM) to perform a deterministic inversion, transforming the video into a latent representation. To maintain temporal consistency, the framework replicates the noise predicted from the first frame across all subsequent frames. This ensures that the added "noise" is structured across the time dimension, preserving the underlying video flow better than independent frame-wise noise.

2. Spatial-Temporal Optimization

During the denoising (cleaning) phase, VideoPure guides the latent variables using a dual-loss optimization strategy:

  • Spatial Loss ($L_{spa}$): This loss increases the distance between the denoised latent and the original adversarial input, effectively "shaking off" pixel-level perturbations.
  • Temporal Loss ($L_{temp}$): Utilizing the RAFT model for optical flow estimation, this loss ensures that adjacent frames remain consistent. This prevents the "flickering" or structural breakdown that typically occurs when frames are purified in isolation.

3. Multi-step Voting

To defend against sophisticated "adaptive" attacks—where an attacker understands the defense mechanism—VideoPure employs a multi-step voting strategy. Instead of relying solely on the final cleaned output, the system extracts intermediate results from various stages of the denoising process. These candidate videos are all processed by the recognition model, which then determines the final class via consensus. This makes it mathematically difficult for an attacker to find a single adversarial direction that fools all versions of the purified video.

Experimental Evaluation

VideoPure was evaluated on two primary benchmarks: UCF-101 (101 action categories) and Kinetics-400 (400 human action classes). Testing involved state-of-the-art video models, including Non-local Networks (NL) and SlowFast, against a variety of threats including PGD, BPDA, EOT (Expectation Over Transformation), and AutoAttack.

Key Results

  • Superior Robustness: On the UCF-101 dataset using a Non-local Network, VideoPure achieved 100% Robust Accuracy against PGD+BPDA attacks. This significantly outperforms the previous state-of-the-art diffusion method, DiffPure DDPM, which reached only 72.3%.
  • Efficiency Gains: VideoPure offers a roughly 4x speedup over previous diffusion-based methods. It can purify a video in approximately 8.5 seconds, compared to the 37.8 seconds required by DiffPure DDPM.
  • The Temporal Discrepancy Advantage: By using different trajectories for adding noise (Temporal DDIM Inversion) and removing it (Guided Denoising), VideoPure creates a "trap" for attackers, making gradient estimation for bypassing the defense extremely difficult.

Conclusion and Future Work

VideoPure represents a significant step forward in securing video recognition systems. By prioritizing temporal consistency through DDIM inversion and optical flow constraints, it effectively nullifies the ability of attackers to hide noise in the temporal gaps between frames.

Despite its success, challenges remain. While 8.5 seconds per video is a major improvement, it is still too slow for real-time applications like autonomous driving or live surveillance. Additionally, the reliance on the RAFT model suggests a potential vulnerability if attackers develop "meta-attacks" targeting optical flow estimation. Future research will focus on further optimizing the diffusion process for real-time speeds and exploring more complex spatial-temporal noise patterns to stay ahead of evolving adversarial strategies.

Summary updated

Abstract

Recent work indicates that video recognition models are vulnerable to adversarial examples, posing a serious security risk to downstream applications. However, current research has primarily focused on adversarial attacks, with limited work exploring defense mechanisms. Furthermore, due to the spatial-temporal complexity of videos, existing video defense methods face issues of high cost, overfitting, and limited defense performance. Recently, diffusion-based adversarial purification methods have achieved robust defense performance in the image domain. However, due to the additional temporal dimension in videos, directly applying these diffusion-based adversarial purification methods to the video domain suffers performance and efficiency degradation. To achieve an efficient and effective video adversarial defense method, we propose the first diffusion-based video purification framework to improve video recognition models’ adversarial robustness: VideoPure. Given an adversarial example, we first employ temporal DDIM inversion to transform the input distribution into a temporally consistent and trajectory-defined distribution, covering adversarial noise while preserving more video structure. Then, during DDIM denoising, we leverage intermediate results at each denoising step and conduct guided spatial-temporal optimization, removing adversarial noise while maintaining temporal consistency. Finally, we input the list of optimized intermediate results into the video recognition model for multi-step voting to obtain the predicted class. We investigate the defense performance of our method against state-of-the-art black-box, gray-box, and adaptive attacks on benchmark datasets and models. Compared with other adversarial purification methods, our method overall demonstrates better defense performance against different attacks. Moreover, our method can be applied as a flexible defense plugin for video recognition models. Our code is available at https://github.com/deep-kaixun/VideoPure.
BibTeX
@article{jiang2025videopure,
  author = {Kaixun Jiang and Zhaoyu Chen and Jiyuan Fu and Lingyi Hong and J Li and Wenqiang Zhang},
  title = {{VideoPure: Diffusion-Based Adversarial Purification for Video Recognition}},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  doi = {10.1109/tcsvt.2025.3579120},
}
Figure