Early Stop And Adversarial Training Yield Better surrogate Model: Very Non-Robust Features Harm Adversarial Transferability

Chaoning Zhang, Gyusang Cho, Philipp Benz, Kang Zhang, Chenshuang Zhang, Chan-Hyun Youn, In So KweonOpenReview 2021 · OpenReview

TL;DR

This paper challenges the conventional wisdom that more robust models lead to better adversarial transferability. Instead, it proposes a "Very Non-Robust Feature" (VNRF) hypothesis, suggesting that these specific features, while exploitable for attacks, actually harm transferability. The authors demonstrate that early stopping training and light adversarial training, which prune these VNRFs, significantly improve adversarial transferability.

What Problem Does This Paper Address?

Transferable Black-box Attacks (TBAs) are a significant security concern, allowing attackers to craft adversarial examples (AEs) on a surrogate model and then use them against a target model without direct access. A key component of TBAs is the transferability of AEs – how well they generalize across different models. Existing research largely attributes this transferability to "non-robust features" (NRFs) learned by models. The prevailing assumption is that longer training, which supposedly learns more NRFs, should lead to better transferability. However, this paper observes a counter-intuitive phenomenon: models trained for fewer epochs (early stopped) exhibit higher adversarial transferability, even though they are less accurate. This discrepancy suggests a more nuanced understanding of feature robustness and its impact on transferability is needed.

Methods

The core of this work lies in proposing and verifying the "Very Non-Robust Feature" (VNRF) hypothesis.

  • VNRF Hypothesis: While NRFs are generally beneficial for transferability, "Very Non-Robust Features" (VNRFs) – features that are highly sensitive to even small perturbations – are hypothesized to harm transferability.
  • Early Stopping: The authors train models and save checkpoints at various epochs. They then evaluate the transferability of AEs generated from these early-stopped models compared to fully trained models.
  • Weight Filtering (L1-norm): To identify VNRFs, they leverage the L1-norm of convolutional filter weights. Filters with high L1-norms are considered more sensitive to input perturbations and thus potential VNRFs. They experiment with zeroing out these filters to see if it improves transferability.
  • Light Adversarial Training: Motivated by the VNRF hypothesis, they propose "light" adversarial training. This technique aims to remove VNRFs while preserving more robust features, contrasting with "heavy" adversarial training that might remove all NRFs, potentially harming performance.

Datasets

The experiments were conducted on two standard image classification datasets:

  • CIFAR-10: Used for training surrogate models and evaluating transferability.
  • ImageNet: Used for evaluating transferability, often with pre-trained models.

The paper also mentions using a dataset from the NeurIPS 2017 adversarial challenge for ImageNet evaluation.

Key Insights and Ideas

  • Early Stopping is Key: The most striking finding is that models trained for fewer epochs (early stopped) yield significantly better adversarial transferability than fully trained models, despite lower accuracy. This directly contradicts the idea that more training always equals better transferability.
  • VNRFs are the Culprits: The paper introduces the VNRF hypothesis, proposing that not all non-robust features are equal. "Very Non-Robust Features" are identified as detrimental to transferability, likely because they are too specific to the surrogate model and don't generalize.
  • Targeted Removal of VNRFs: The L1-norm filtering experiment provides empirical evidence for the VNRF hypothesis. Zeroing out filters with high L1-norms (suspected VNRFs) demonstrably improves transferability.
  • Light Adversarial Training for the Win: This insight leads to a practical technique: light adversarial training. By carefully tuning the adversarial training process, they can remove VNRFs while retaining beneficial NRFs, leading to substantial improvements in transferability.

Key Results

  • Early Stopping Boost: On CIFAR-10, early stopping improved transfer rates to over 98% for models like VGG16 and up to 99.9% for ResNet18. On ImageNet, early stopping also non-trivially improved transferability.
  • VNRF Hypothesis Verification: Zeroing out filters with high L1-norms consistently improved transferability on CIFAR-10.
  • Light Adversarial Training Effectiveness:
    • On ImageNet, light adversarial training (with a small perturbation budget, e.g., $\epsilon=0.05$) improved transfer rates from 76.4% to 96.7% for I-FGSM on ResNet18.
  • For non-CNN targets like ViT-B/16, transfer rates increased from 19.4% to 87.4% with light adversarial training.
  • In the more challenging targeted setting, transfer rates increased from 0.4% to 38.4% for I-FGSM on ResNet18.

Limitations and Future Work

  • Early Stop vs. Adversarial Training: The paper notes that early stopping's benefit on transferability is less pronounced for adversarially trained models, as they already focus on robust features.
  • VNRF Definition Nuance: While the L1-norm filtering provides evidence, a more precise definition and identification of VNRFs could be explored.
  • Transfer Learning Connection: The authors suggest that their findings on VNRFs harming transferability might also shed light on similar phenomena observed in transfer learning to new datasets, leaving this as a future research direction.
  • Generalizability of Techniques: While effective on common architectures, further investigation into the generalizability of early stopping and light adversarial training across a wider range of models and tasks would be valuable.

Summary updated

Abstract

The transferability of adversarial examples (AE); known as adversarial transferability, has attracted significant attention because it can be exploited for TransferableBlack-box Attacks (TBA). Most lines of works attribute the existence of the non-robust features improves the adversarial transferability. As a motivating example, we test the adversarial transferability on the early stopped surrogate models, which are known to be concentrated on robust features than non-robust features from prior works. We find that the early stopped models yield better adversarial transferability than the models at the final epoch, which leaves non-intuitive interpretation from the perspective of the robust and non-robust features (NRFs). In this work, we articulate a novel Very Non-Robust Feature(VNRF) hypothesis that the VNRFslearned can harm the adversarial transferability to explain this phenomenon. This hypothesis is partly verified through zeroing some filters with highl1norm values. This insight further motivates us to adopt light adversarial training that mainly removes the VNRFs for significantly improving the transferability.
BibTeX
@misc{zhang2021early,
  author = {Chaoning Zhang and Gyusang Cho and Philipp Benz and Kang Zhang and Chenshuang Zhang and Chan-Hyun Youn and In So Kweon},
  title = {{Early Stop And Adversarial Training Yield Better surrogate Model: Very Non-Robust Features Harm Adversarial Transferability}},
  year = {2021},
  url = {https://openreview.net/forum?id=ECC7T-torK},
  howpublished = {OpenReview},
}

Loading related content...

Figure