Everywhere Attack: Attacking Locally and Globally to Boost Targeted Transferability

Hui Zeng, Sanshuai Cui, Biwei Chen, Anjie PengAAAI 2025 · DOI

TL;DR

This paper introduces an "everywhere attack" that significantly boosts the success rate of targeted adversarial attacks by optimizing for multiple target objects across different image regions. This approach effectively overcomes the challenge of attention mismatch between different neural network models, leading to improved transferability for existing attack methods.

What Problem Does This Paper Address?

Adversarial examples (AEs) are intentionally crafted inputs designed to fool deep neural networks (DNNs). A key property of AEs is their "transferability" – AEs created for one model can often fool other, unseen models. While untargeted transferability (making a model misclassify an image) has seen significant progress, targeted transferability (making a model classify an image as a specific incorrect class) remains a much harder problem. This difficulty stems from the fact that different models often focus their attention on different regions of an image when trying to identify a specific target class. This "attention mismatch" causes AEs that work well on a surrogate model to fail when transferred to a different victim model.

Methods

The core innovation is the "everywhere attack" scheme. Instead of optimizing for a single, high-confidence target object in the entire image (as in previous methods), this approach divides the victim image into non-overlapping blocks. For each block, it synthesizes a target object, effectively creating an "army of targets" distributed across the image. The idea is that by placing targets in many local regions, at least one of these targets is likely to fall within the attention area of the victim model, regardless of where it focuses. This method is designed to be "method-agnostic," meaning it can be easily integrated with existing transferable attack algorithms to enhance their performance. The paper also details how this scheme can be applied to various iterative attack methods, such as CE, Logit, and Margin attacks.

Datasets

The experiments were conducted on the ImageNet-compatible dataset, which consists of 1000 images.

Key Insights and Ideas

  • Attention Mismatch is Key: The paper identifies the significant difference in attention regions between surrogate and victim models as the primary bottleneck for targeted transferability.
  • "Army of Targets" Strategy: The novel idea of optimizing for multiple, distributed target objects rather than a single one is a paradigm shift for boosting transferability.
  • Method Agnosticism: The proposed "everywhere" scheme can be readily combined with various existing attack techniques, amplifying their effectiveness without requiring fundamental changes to the base attack.
  • Improved Coverage: The "everywhere" attack demonstrably increases the overlap between the attention maps of surrogate and victim models, directly addressing the attention mismatch problem.

Key Results

  • The "everywhere attack" universally improves the transferability of existing state-of-the-art targeted attacks.
  • For the widely adopted Logit attack, transferability was improved by a significant margin, ranging from 28.8% to 300%.
  • The method showed strong performance even when transferring to transformer-based models like Swin, which are often more challenging.
  • Evaluation on the real-world Google Cloud Vision API confirmed the superiority of the proposed method, demonstrating its practical applicability.

Limitations and Future Work

  • Computational Cost: While the "everywhere" scheme enhances transferability, it inherently increases computational cost due to the optimization across multiple regions. The paper notes that the parameter N (number of sampled blocks) can be used to balance attack power and computational efficiency.
  • Generative vs. Iterative Attacks: The paper compares its iterative approach with generative methods, suggesting that while iterative attacks with the "everywhere" scheme can achieve comparable or better transferability, generative attacks offer higher computational efficiency once generators are trained. Future work could explore more efficient ways to integrate the "everywhere" concept into generative models.
  • Hyperparameter Tuning: The supplementary material is mentioned for ablation studies on key hyperparameters, indicating that their optimal selection is crucial for performance.

Summary updated

BibTeX
@inproceedings{zeng2025everywhere,
  author = {Hui Zeng and Sanshuai Cui and Biwei Chen and Anjie Peng},
  title = {{Everywhere Attack: Attacking Locally and Globally to Boost Targeted Transferability}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.33061},
}
Figure