Learning Transferable Adversarial Perturbations

Krishna kanth Nakka, Mathieu SalzmannNeurIPS 2021 · OpenReview

Deep neural networks (DNNs) are notoriously vulnerable to adversarial attacks—imperceptible tweaks to an image that cause a model to make incorrect predictions. While traditional iterative attacks (like PGD) are highly effective, they are computationally expensive because they optimize the perturbation for each specific image. Generative attacks solve this by using a secondary neural network to instantly output a perturbation in a single forward pass.

However, generative attacks suffer from a major flaw: they tend to overfit to the specific architecture, dataset, or task they were trained on. If an attacker operates in a black-box scenario—lacking knowledge of the exact target model or access to the target training data—the effectiveness of existing generative attacks plummets. This paper tackles this gap by asking: How can we train a perturbation generator that generalizes across unknown architectures, unseen data distributions, and entirely different computer vision tasks?

The Feature Separation Loss Approach

Standard generative attacks train the generator using a cross-entropy loss that pushes the perturbed image across the target model's final decision boundary. Because the final layers of a network are highly task-specific, these attacks fail to transfer well to other models or tasks.

To overcome this, the authors propose a Feature Separation Loss. Instead of looking at the final prediction, they feed both the clean image and the generated adversarial image into a pretrained classifier and extract the feature maps at an intermediate (mid-level) layer. The generator is then trained to maximize the $L_2$ distance (Frobenius norm) between the clean and adversarial feature maps at this specific layer.

Overview of the proposed transferable adversarial perturbation framework

By freezing the target classifier and only updating the generator's weights to disrupt these intermediate representations, the generator learns to produce structured, quasi-imperceptible patterns. These patterns scramble the network's internal logic long before the data reaches the highly specialized final classification layers.

Key Insights: The "Goldilocks Zone" of Features

The authors discovered that mid-level layers are the perfect target for transferable attacks. Bottom layers (which process basic edges and colors) require massive, visible perturbations to disrupt. Top layers (which process object parts) are too specific to the training task. Mid-level layers, however, process textures and patterns that are fundamentally shared across diverse CNN architectures and tasks.

Interestingly, the generated perturbations aren't just random noise. They visually manifest as repeating textures that strongly correlate with the specific mid-level filters being disrupted.

Visualizations of intermediate layer filters showing repeating textures

Furthermore, while training the generator on proxy datasets that share visual similarities with the target data yields incredibly high transferability, it isn't strictly necessary. Even with a massive domain gap—such as training on Chest X-rays to attack ImageNet—targeting mid-level features still produces surprisingly effective attacks.

Experimental Results

The proposed method was evaluated across various scenarios, demonstrating significant improvements over state-of-the-art baselines like GAP and CDA:

  • Standard Black-Box: The method achieved an average fooling rate of 86.8% across various ImageNet models (VGG, ResNet, Inception, DenseNet, SqueezeNet), outperforming baselines by 10.5 to 13.5 percentage points.
  • Strict Black-Box (No Target Data): When trained on a synthetic "Comics" dataset and tested on ImageNet models, the method achieved a 90.8% average fooling rate, vastly outperforming GAP (76.9%) and CDA (80.5%).
  • Extreme Cross-Domain: Training a generator on Chest X-rays to attack ImageNet models still achieved a 68.5% average fooling rate.
  • Cross-Task (Classification to Detection): A generator trained to fool an image classifier successfully transferred to fool an SSD object detector. For example, attacking a VGG16-based SSD detector dropped its Mean Average Precision (mAP) from 68.1 down to 10.5.

Limitations and Future Directions

While highly effective, the current method attacks a single intermediate layer at a time, requiring an empirical sweep to find the optimal layer (though this optimal layer tends to be independent of the target model). Additionally, while the method successfully bypasses some standard defenses, it struggles against highly robust, computationally heavy defenses like Feature Denoising (FD) and Projected Gradient Descent (PGD) adversarial training.

Future work will likely explore attacking ensembles of layers and using these insights into mid-level feature vulnerabilities to design better, more robust black-box defense mechanisms.

Summary updated

Abstract

While effective, deep neural networks (DNNs) are vulnerable to adversarial attacks. In particular, recent work has shown that such attacks could be generated by another deep network, leading to significant speedups over optimization-based perturbations. However, the ability of such generative methods to generalize to different test-time situations has not been systematically studied. In this paper, we, therefore, investigate the transferability of generated perturbations when the conditions at inference time differ from the training ones in terms of the target architecture, target data, and target task. Specifically, we identify the mid-level features extracted by the intermediate layers of DNNs as common ground across different architectures, datasets, and tasks. This lets us introduce a loss function based on such mid-level features to learn an effective, transferable perturbation generator. Our experiments demonstrate that our approach outperforms the state-of-the-art universal and transferable attack strategies.
BibTeX
@inproceedings{nakka2021learning,
  author = {Krishna kanth Nakka and Mathieu Salzmann},
  title = {{Learning Transferable Adversarial Perturbations}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=sIDvIyR5I1R},
}
Figure