Generative Dynamic Patch Attack

Xiang Li, Shihao Ji

TL;DR

This paper introduces Generative Dynamic Patch Attack (GDPA), a novel method that generates both the pattern and location of adversarial patches dynamically for each input image. GDPA not only achieves higher attack success rates than existing patch attacks but also significantly improves model robustness when used for adversarial training, outperforming previous defense methods.

What Problem Does This Paper Address?

Deep neural networks (DNNs), while powerful, are vulnerable to adversarial attacks. Patch attacks, which involve altering a specific region of an image, are particularly concerning because they can be physically realizable (e.g., stickers on signs). Existing patch attack methods often use fixed or random patch locations, which are suboptimal for both attack and, crucially, for adversarial training. Models trained with these methods are not robust enough against sophisticated patch attacks. This paper addresses the limitation of static or random patch placement by proposing a method that dynamically determines the optimal patch pattern and location for each image, leading to more effective attacks and more robust defenses.

Methods

The core of the proposed method is the Generative Dynamic Patch Attack (GDPA) framework. GDPA is an end-to-end differentiable system that generates an adversarial patch by:

  • Image-Dependent Patch Generation: Unlike previous methods, GDPA's generator learns to produce both the patch pattern and its location specifically for each input image. This is based on the intuition that different images have different "weak spots" that DNNs are sensitive to.
  • Encoder-Decoder Architecture: A shared encoder extracts features from the input image, which are then fed into two separate decoders: one for generating the patch location (lx, ly) and another for generating the patch pattern.
  • Differentiable Affine Transformation: To inject the generated patch into the image, GDPA uses a differentiable affine transformation, specifically a translation, combined with bilinear interpolation. This ensures the entire pipeline is end-to-end differentiable, allowing for gradient-based optimization.
  • Weighted Adversarial Patch Injection: The patch is injected using a weighted combination of the original image and the adversarial patch, controlled by a visibility parameter $\alpha$. This allows for generating both visible and nearly invisible patches.
  • Adversarial Training (GDPA-AT): GDPA is integrated into an adversarial training framework. The generator (G) and the target classifier (T) are trained iteratively in a minimax game: G tries to maximize the classification loss of T by generating adversarial patches, while T tries to minimize this loss by learning to be robust against these patches.

Datasets

The authors evaluated their method on three benchmark datasets:

  • VGGFace: A large-scale dataset for face recognition.
  • Traffic Sign: A dataset containing images of traffic signs.
  • ImageNet: A large-scale object recognition benchmark.

For adversarial training and defense evaluation, they also used specific subsets and configurations of these datasets, as well as standard adversarial attack datasets for comparison.

Key Insights and Ideas

  • Dynamic Patch Placement is Crucial: The paper strongly suggests that the location of an adversarial patch matters significantly. Learning image-specific locations leads to much higher attack success rates than using fixed or random locations.
  • End-to-End Differentiability Enables Better Optimization: By making the entire patch generation and injection process differentiable, GDPA can be optimized more effectively, leading to stronger attacks and more robust defenses.
  • GDPA as a Generic Framework: The method is flexible and can generate various types of patches (dynamic/static, visible/invisible) with minor configuration changes.
  • Adversarial Training with GDPA is Highly Effective: GDPA-AT not only defends against patch attacks but also shows significant robustness against conventional adversarial attacks (like PGD), outperforming existing defense methods.
  • Inference Speed Advantage: GDPA's generator produces patches in a single forward pass, making it significantly faster (40-50x) than iterative optimization-based attack methods.

Key Results

  • Attack Performance: GDPA consistently achieved higher Attack Success Rates (ASRs) than state-of-the-art patch attack algorithms (LAVAN, ROA) across VGGFace, Traffic Sign, and ImageNet datasets, especially with larger patch sizes. For example, on VGGFace with 10% pixels attacked, GDPA achieved 99.5% ASR compared to ROA-Exh's 99.3%.
  • Defense Performance: Models trained with GDPA-AT (GDPA-AT) demonstrated superior robustness against high-profile patch attacks (eyeglasses, sticker, LAVAN) compared to PGD-AT and DOA. GDPA-AT also showed significant robustness against conventional PGD attacks, outperforming DOA and being comparable to PGD-AT.
  • Inference Speed: GDPA was approximately 40-50 times faster than PGD and ROA for generating attacks on the VGGFace dataset.
  • Visibility Control: The parameter $\alpha$ effectively controls the visibility of the patch, allowing for trade-offs between attack strength and imperceptibility.

Limitations and Future Work

  • Hyperparameter Tuning: The paper notes that parameters like $\beta$ (controlling the tanh slope for location generation) require careful tuning and can impact training difficulty.
  • Advanced Affine Transformations: While the current GDPA uses translation, the authors mention that learning more advanced affine transformations (rotation, dilation, shear) could further improve performance and is a direction for future work.
  • Generalization to Other Attack Types: While GDPA-AT shows robustness against conventional attacks, further exploration into its effectiveness against a wider range of adversarial perturbations could be beneficial.

Summary updated

Abstract

Adversarial patch attack is a family of attack algorithms that perturb a part of image to fool a deep neural network model. Existing patch attacks mostly consider injecting adversarial patches at input-agnostic locations: either a predefined location or a random location. This attack setup may be sufficient for attack but has considerable limitations when using it for adversarial training. Thus, robust models trained with existing patch attacks cannot effectively defend other adversarial attacks. In this paper, we first propose an end-to-end patch attack algorithm, Generative Dynamic Patch Attack (GDPA), which generates both patch pattern and patch location adversarially for each input image. We show that GDPA is a generic attack framework that can produce dynamic/static and visible/invisible patches with a few configuration changes. Secondly, GDPA can be readily integrated for adversarial training to improve model robustness to various adversarial attacks. Extensive experiments on VGGFace, Traffic Sign and ImageNet show that GDPA achieves higher attack success rates than state-of-the-art patch attacks, while adversarially trained model with GDPA demonstrates superior robustness to adversarial patch attacks than competing methods.
BibTeX
@misc{li2021generative,
  author = {Xiang Li and Shihao Ji},
  title = {{Generative Dynamic Patch Attack}},
  year = {2021},
  eprint = {2111.04266},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.04266},
}
Figure