On the Role of Generalization in Transferability of Adversarial Examples

Yilin Wang, Farzan Farnia

Adversarial examples—inputs specifically crafted to deceive neural networks—pose a significant security risk in machine learning. A particularly concerning property of these examples is transferability: an attack designed to fool one model (the substitute) often succeeds against another (the target), even if the target's architecture and training data are unknown to the attacker. This phenomenon forms the backbone of "black-box" adversarial attacks.

While the existence of transferability is well-documented, the underlying factors that govern its effectiveness remain a subject of intense research. This paper explores the hypothesis that a substitute model's ability to generalize to unseen data is a primary driver of adversarial transferability. The authors argue that if a model overfits its training data, the resulting adversarial perturbations become "overfitted" to that specific model's idiosyncrasies, failing to exploit the fundamental structural weaknesses of the data distribution that would affect other models.

Theoretical Foundations

The authors approach the problem through the Adversarial Example Game (AEG), a max-min framework where an adversary attempts to find an attack strategy that maximizes prediction error against the most robust possible classifier. This framework allows for the derivation of theoretical generalization bounds for attacks.

Central to their theory is the concept of spectral capacity, defined as the product of the spectral norms of a network's weight matrices. Their mathematical analysis suggests that lower spectral norms—which correspond to smoother, Lipschitz-continuous functions—lead to a smaller gap between attack success on training data versus test data. In essence, "smoother" models produce adversarial examples that are more likely to represent general features of the data rather than model-specific noise.

Experimental Methodology

To validate their theory, the authors employed two primary regularization techniques to improve substitute model generalization:

  1. Spectral Normalization (Explicit): Constraining the $L_2$-operator norm of weight matrices to a hyperparameter $\beta$.
  2. Early Stopping (Implicit): Halting training at the point of peak validation performance to prevent the model from learning noise in the training set.

Experiments were conducted using standard benchmarks: CIFAR-10, CIFAR-100, and SVHN. AlexNet and Inception-Net served as substitute models, while VGG-16 and ResNet-18 were used as unseen target models. Attacks were generated using the Fast Gradient Method (FGM) and Projected Gradient Descent (PGD).

Key Findings

The Impact of Spectral Regularization

The experiments demonstrated a clear inverse relationship between the substitute model's generalization error and the transferability of its adversarial examples. As the spectral normalization parameter $\beta$ is decreased (increasing regularization), the model generalizes better, and the transferability of its attacks increases.

Figure 1: Adversarial generalization error increases as spectral regularization (β) is relaxed, indicating poorer generalization.

As shown in the figures, relaxing the regularization (increasing $\beta$) leads to a rise in adversarial generalization error. Conversely, the transferability score—the success rate of the attack on an unseen target—drops as the substitute model begins to overfit.

Figure 4: Transferability score to a ResNet-18 target decreases as the substitute model's regularization is weakened (higher β).

Specifically, applying spectral normalization ($\beta=1.0$) to an AlexNet substitute on CIFAR-10 increased the transferability rate to a VGG-16 target from 10.5% to 16.2% for PGD attacks.

Early Stopping and Structural Weaknesses

Early stopping proved to be an even more potent tool. For an Inception-Net substitute on CIFAR-10, early stopping reduced the generalization error from 0.517 to 0.073 while simultaneously boosting transferability to ResNet-18 from 10.4% to 17.2%.

Qualitative observations supported these metrics. Perturbations from regularized models appeared less noisy and more focused on the central semantic objects in the images. This suggests that regularization forces the adversary to find "structural" weaknesses in the data distribution rather than exploiting "noise-like" features unique to the substitute model.

Conclusion

This research establishes a robust link between generalization and transferability. By improving a substitute model's generalization through Lipschitz continuity and early stopping, researchers can generate significantly more effective black-box adversarial attacks. These findings suggest that transferability is not just a quirk of deep learning, but a direct consequence of how well a model captures the underlying distribution of its data. Future work will likely explore how other regularization techniques, such as Dropout or Batch Normalization, influence this critical relationship.

Summary updated

BibTeX
@inproceedings{wang2023role,
  author = {Yilin Wang and Farzan Farnia},
  title = {{On the Role of Generalization in Transferability of Adversarial Examples}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=g6zcauIP17r},
}
Figure