Incorporating Locality of Images to Generate Targeted Transferable Adversarial Examples

Zhipeng Wei, Jingjing Chen, Zuxuan Wu, Yu-Gang JiangarXiv 2022 · OpenReview

Abstract

Targeted adversarial attacks, which aim to deceive machine learning models into predicting a specific incorrect class, are notoriously difficult to transfer between different model architectures. This paper introduces Self-Universality (SU), a novel framework that enhances the transferability of targeted attacks by exploiting the internal locality of images. By forcing adversarial perturbations to be effective across various local regions of a single image, the method captures fundamental features of the target class. Experimental results on the ImageNet dataset demonstrate that SU achieves a 12% improvement in Targeted Attack Success Rate (TASR) over existing state-of-the-art techniques without requiring external training data.

Introduction

Adversarial attacks involve making subtle, often imperceptible changes to an image to trick a machine learning model. While non-targeted attacks (simply causing a misclassification) transfer relatively well between models, targeted attacks (forcing a specific wrong prediction, such as a "cat" being classified as a "toaster") are much harder to transfer.

The primary challenge is that different models "look" at images differently; a perturbation optimized for one model might not align with the decision boundaries of another. Previous solutions attempted to solve this by training auxiliary "helper" networks on large external datasets to learn universal target features. However, this approach is computationally expensive and relies on the availability of massive training sets, which is often impractical in real-world scenarios.

Methodology: The Self-Universality Framework

The authors propose the concept of Self-Universality (SU), based on the observation that perturbations effective across multiple images (universal perturbations) transfer better. SU achieves a similar effect using only a single image by treating different crops of that image as independent samples.

The Self-Universality (SU) attack framework, illustrating the dual-stream optimization and feature similarity loss.

As shown in the framework above, the SU attack utilizes several key components:

  1. Dual-Stream Optimization: The attack simultaneously processes a "Global Input" (the full image) and a "Local Input" (a randomly cropped and resized patch).
  2. Shared Perturbations: The same adversarial noise is applied to both the global and local views.
  3. Feature Similarity Loss: In addition to standard classification loss (Cross-Entropy), the method employs a Cosine Similarity loss. This forces the internal feature representations of the global adversarial image and the local adversarial patch to converge, ensuring the adversarial noise becomes the dominant feature.
  4. Plug-and-Play Integration: SU is designed to be easily combined with existing momentum-based and input-diversity techniques (collectively referred to as DTMI).

Experimental Setup

The researchers evaluated the SU method using the ImageNet-compatible dataset from the NIPS 2017 Competition on Adversarial Attacks and Defenses. The dataset includes 1,000 images with specific target labels. Testing was conducted across four diverse architectures: ResNet50, DenseNet121, VGG16, and Inception-v3.

Key Results and Insights

The SU method demonstrates that universality can be achieved through "self-sampling." By maximizing feature similarity between global and local views, the attack ensures that the adversarial noise captures the fundamental characteristics of the target class that are shared across different model architectures.

Comparison of Targeted Attack Success Rate (TASR) and gradient norm across iterations for different attack variants.

Key findings include:

  • Significant Performance Boost: On the ImageNet-compatible dataset, the SU attack yielded a 12% improvement in TASR compared to previous state-of-the-art methods.
  • Ensemble Effectiveness: When attacking an ensemble of models, the SU method reached success rates as high as 75.3% for certain target models.
  • Convergence and Stability: As illustrated in the gradient norm analysis, while SU requires more iterations to converge (approximately 300), it effectively avoids the "vanishing gradient" problem that often hinders standard targeted attacks.
  • Efficiency: Unlike GAN-based or auxiliary-classifier methods, SU only requires two forward passes per iteration, making it significantly more computationally efficient.

Conclusion

The Self-Universality framework provides a powerful, data-efficient way to generate transferable targeted adversarial examples. By exploiting image locality, it bridges the gap between white-box and black-box targeted attacks. Future work may explore more complex data augmentations or different types of "locality" to further refine the universality of generated perturbations.

Summary updated

BibTeX
@misc{wei2022incorporating,
  author = {Zhipeng Wei and Jingjing Chen and Zuxuan Wu and Yu-Gang Jiang},
  title = {{Incorporating Locality of Images to Generate Targeted Transferable Adversarial Examples}},
  year = {2022},
  url = {https://openreview.net/forum?id=PYoQvEfMwU4},
}
Figure