Transferable Perturbations of Deep Feature Distributions

Nathan Inkawhich, Kevin J. Liang, Lawrence Carin, Yiran Chen

Adversarial attacks—tiny, often invisible perturbations added to an image to fool a computer vision model—are notoriously difficult to execute when the attacker lacks access to the target model's architecture. These "blackbox" attacks are especially challenging when they are targeted (e.g., forcing a model to classify a "dog" specifically as a "toaster").

Most existing transfer attacks rely on perturbing the final output layer of a known "substitute" model, hoping the adversarial example transfers to the unknown target. However, final layers are often highly specialized to a specific model's architecture, causing attacks to fail when crossing between different network families (like ResNet to VGG).

This paper introduces a powerful new method called the Feature Distribution Attack (FDA), which achieves state-of-the-art results in targeted blackbox attacks by targeting the internal feature distributions of deep neural networks rather than their final outputs.

The Feature Distribution Attack (FDA)

Instead of looking at the final label output, FDA targets the intermediate layers of a neural network. The process involves two main steps:

  1. Auxiliary Modeling: For a given whitebox source model, the researchers train small "auxiliary" neural networks to recognize class-specific patterns at various internal layers. These auxiliary models estimate the probability that a set of intermediate features belongs to a specific target class.
  2. The Attack: To turn a "dog" into a "toaster," the input image is perturbed until the auxiliary model at a specific intermediate layer is convinced the internal features look like a "toaster."

The authors propose three variants of this attack:

  • FDA: The base version that maximizes the target class probability at the intermediate layer.
  • FDA+ms (Minimize Source): Simultaneously pushes the features away from the original source class distribution.
  • FDA+fd (Feature Disruption): Maximizes the distance from the original feature vector, essentially "scrambling" the original identity while moving toward the target.

The Inflection Point: Task-Specific vs. Model-Specific Features

A core insight of this research is the distinction between general and specialized features. Early layers in a network learn general features (like edges and textures) that are "task-specific"—meaning they are common to almost all models trained on a dataset like ImageNet. Later layers, however, perform extra processing to suit the specific architecture, becoming "model-specific."

Correlation of feature distributions across layer depth. The sharp increase indicates where features transition from general to highly specialized, marking the ideal inflection point for attacks.

The researchers identified an "inflection point" in network depth where features are mature enough to represent a semantic class but haven't yet become overfit to the specific model. Attacking at this specific intermediate layer yields the highest transferability to unseen blackbox models.

Key Results and Performance

The FDA method, particularly the FDA+fd variant, significantly outperforms existing state-of-the-art transfer attacks.

Targeted success rate (tSuc) and Error vs. Depth for various attacks. FDA variants (especially FDA+fd in pink) show a distinct peak at intermediate layers, significantly outperforming baselines.

When transferring an attack from a DenseNet-121 source model to a ResNet-50 target model, the FDA+fd variant achieved a 30.9% targeted success rate. In stark contrast, standard momentum-based attacks achieved only a 2.2% success rate.

Furthermore, the optimal layer for transferring attacks was found to be largely model-agnostic. For instance, if layer 7 is the optimal depth for attacking from DenseNet to VGG, it is typically also the best depth for attacking from DenseNet to ResNet. Successful FDA attacks cause massive disruption throughout the entire depth of the target blackbox model, unlike standard attacks that often only disrupt the very last layers.

Limitations and Future Directions

While highly effective, FDA comes with computational overhead. The method requires training auxiliary classifiers for every layer and class an attacker wishes to target. Although the authors demonstrated that training on a subset of representative classes generalizes well, scaling this comprehensively remains resource-intensive.

Additionally, this study focused on undefended Convolutional Neural Networks (CNNs). Future work will need to explore how these intermediate feature perturbations perform against models fortified with adversarial defenses, as well as how these "inflection points" manifest in newer architectures like Vision Transformers (ViTs).

Summary updated

Abstract

We show that perturbations based-on intermediate feature distributions yield more transferable adversarial examples and allow for analysis of the affects of adversarial perturbations on intermediate representations.
BibTeX
@inproceedings{inkawhich2020transferable,
  author = {Nathan Inkawhich and Kevin J. Liang and Lawrence Carin and Yiran Chen},
  title = {{Transferable Perturbations of Deep Feature Distributions}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=POFpQaVawm},
}
Figure