Learning Universal Adversarial Perturbations with Generative Models

Jamie Hayes, George DanezisIEEE Symposium on Security and Privacy Workshops 2018 · OpenReview

TL;DR

This paper introduces Universal Adversarial Networks (UANs), a novel generative approach to creating "universal adversarial perturbations" (UAPs) – single image modifications that can fool a target neural network on any input. UANs outperform existing UAP generation methods and demonstrate robustness and transferability across different models.

What Problem Does This Paper Address?

Deep neural networks, despite their impressive performance, are notoriously vulnerable to adversarial examples. These are subtly altered inputs that are visually indistinguishable to humans but cause the model to misclassify them. While most research focuses on crafting adversarial examples for specific inputs, a more concerning threat is the existence of "universal adversarial perturbations" (UAPs). A UAP is a single perturbation that, when applied to any input from a dataset, has a high probability of causing a misclassification. This poses a significant security risk, as a single, pre-computed perturbation could be used to attack a wide range of inputs. The challenge is to efficiently and effectively generate such UAPs that are robust and generalizable.

Methods

The core contribution is the Universal Adversarial Network (UAN), a generative model designed to learn UAPs. The UAN takes a random noise vector as input and outputs a perturbation $\delta$. This perturbation is then scaled and added to a clean input image $x$, creating an adversarial image $x + \delta'$. The UAN is trained by optimizing a loss function that aims to:

  1. Fool the target model: Maximize the probability of misclassification (or a specific target class for targeted attacks).
  2. Minimize perturbation norm: Keep the perturbation $\delta'$ visually imperceptible, typically by minimizing its $l_p$ norm (e.g., $l_2$ or $l_\infty$).

This approach differs from previous UAP methods by leveraging a generative model, allowing it to learn a distribution of perturbations rather than relying on iterative optimization for each input or a fixed perturbation. The paper also explores both non-targeted (any misclassification) and targeted (specific misclassification) attacks.

Datasets

The authors evaluate their UANs on two standard datasets:

  • CIFAR-10: A dataset of 60,000 32x32 RGB images across 10 classes.
  • ImageNet: A larger dataset of 50,000 224x224 RGB images across 1,000 classes.

They use pre-trained models on these datasets as their "target models" to attack.

Key Insights and Ideas

  • Generative models can learn UAPs: The paper demonstrates that a generative network (UAN) can effectively learn to produce universal adversarial perturbations, a task previously tackled with iterative optimization methods.
  • UAPs are a potent threat: The success rates achieved by UANs highlight the significant vulnerability of even robust models to these single, general-purpose perturbations.
  • Transferability is key: UAPs generated by a UAN trained on one model can effectively fool other, unseen models, underscoring their broad applicability in black-box attack scenarios.
  • Generalizability of UAPs: While UANs learn to generate UAPs, they don't produce identical perturbations for every input. This suggests a learned distribution of perturbations that can generalize, rather than a single fixed pattern.
  • Defense limitations: Adversarial training, a common defense, is effective against standard adversarial examples but UANs can still fool adversarially trained models, albeit with reduced success.

Key Results

  • Improved performance: UANs consistently outperform state-of-the-art methods (Moosavi-Dezfooli et al. [19] and Mopuri et al. [21]) in generating UAPs, achieving lower error rates across various target models and datasets. For example, on CIFAR-10 with $l_\infty$ metric, UAN achieved error rates as low as 0.666 compared to 0.832 for Moosavi-Dezfooli et al. on VGG-19.
  • High transferability: UAPs crafted by a UAN trained on VGG-19 achieved an error rate of 61.2% when tested on ResNet-101, demonstrating strong cross-model effectiveness.
  • Ensemble attacks: UANs can be trained to fool an ensemble of target models simultaneously, achieving comparable error rates to single-model attacks.
  • Robustness to training data size: The performance of UANs is not significantly impacted by the amount of training data available, suggesting that knowledge of the dataset's structure is more critical than the sheer volume of samples.
  • Targeted attacks show structure: For targeted attacks, the generated UAPs often exhibit patterns related to the target class (e.g., a golf ball pattern for a golf ball target).

Limitations and Future Work

  • White-box assumption: The current UAN attack assumes white-box access to the target model (i.e., knowledge of its architecture and weights) for backpropagation. Future work could explore black-box UAP generation.
  • Real-world limitations: The paper acknowledges that the assumption of full access to training data samples for the attacker is a simplification and not always realistic.
  • Adversarial training mitigation: While UANs can still fool adversarially trained models, the success rate is reduced. Further research could explore more robust defenses against UANs.
  • Generalizability vs. Attack Success: The paper investigates whether learning to generate multiple UAPs impacts attack accuracy, concluding that it does not significantly reduce accuracy. However, the exploration of the learned distribution of UAPs could be further expanded.

Summary updated

Abstract

Neural networks are known to be vulnerable to adversarial examples, inputs that have been intentionally perturbed to remain visually similar to the source input, but cause a misclassification. It was recently shown that given a dataset and classifier, there exists so called universal adversarial perturbations, a single perturbation that causes a misclassification when applied to any input. In this work, we introduce universal adversarial networks, a generative network that is capable of fooling a target classifier when it's generated output is added to a clean sample from a dataset. We show that this technique improves on known universal adversarial attacks.
BibTeX
@inproceedings{hayes2018learning,
  author = {Jamie Hayes and George Danezis},
  title = {{Learning Universal Adversarial Perturbations with Generative Models}},
  booktitle = {IEEE Symposium on Security and Privacy Workshops},
  year = {2018},
  url = {https://openreview.net/forum?id=xkVQPMHgsPc},
}
Figure