Intriguing Properties of Adversarial Training at Scale

Cihang Xie, Alan L. YuilleICLR 2020 · OpenReview

TL;DR

This paper identifies two major bottlenecks in large-scale adversarial training on ImageNet: the "identity crisis" of Batch Normalization when mixing clean and adversarial data, and a surprising "thirst" for network depth. By disentangling normalization statistics and pushing models to unprecedented scales (like ResNet-638), the authors significantly boost adversarial robustness.

What Problem Does This Paper Address?

Adversarial training (AT)—training a model on examples perturbed by an attacker—is the gold standard for building robust neural networks. However, while it works well on small datasets like CIFAR-10, scaling it to ImageNet has proven difficult and often yields underwhelming results.

The authors investigate why standard deep learning "best practices" seem to fail in the adversarial context. Specifically, they tackle two questions:

  1. The Normalization Paradox: Why does Batch Normalization (BN), which usually helps models converge, seem to hinder robustness?
  2. The Capacity Ceiling: Why does increasing model depth (e.g., moving from ResNet-50 to ResNet-152) provide only marginal gains in robustness compared to the massive gains seen in standard accuracy?

Methods

The authors conduct a rigorous diagnosis of AT strategies using a unified framework on ImageNet. Their novel approaches include:

  • Mixture Batch Normalization (MBN): Based on the hypothesis that clean and adversarial images belong to different domains, MBN uses separate BN layers to estimate statistics for each. This "disentangles" the two distributions during training.
Diagram comparing standard Batch Normalization with Mixture Batch Normalization (MBN)
  • Extreme Scaling: To test the limits of network capacity, the authors move beyond standard architectures, training models as deep as ResNet-638.
  • Running Statistics Heuristic: They found that BN behaves inconsistently between training (using batch stats) and testing (using running stats). They propose a heuristic of using the running statistics from the final 10 epochs of training to ensure consistency.
  • Adversarial Logits Pairing (ALP) Re-evaluation: They provide a cleaner implementation of ALP, showing that while it helps, it is not the primary driver of robustness at scale.

Datasets

The study focuses exclusively on ImageNet, the standard large-scale benchmark for computer vision. This is significant because most prior adversarial research was confined to smaller datasets where architectural bottlenecks are less apparent.

Key Insights and Ideas

  • The Two-Domain Hypothesis: The most surprising finding is that for models using standard BN, simply removing clean images from the training set boosts adversarial robustness by 18.3%. This suggests that clean and adversarial images have such different statistical distributions that forcing them through the same BN layer creates a "mixture distribution" that the model cannot effectively normalize.
Graph showing adversarial robustness peaking when the ratio of clean images in training is reduced to 0%
  • Adversarial Training is "Harder" than Classification: In standard training, accuracy plateaus quickly as you add layers (e.g., the difference between ResNet-152 and ResNet-200 is tiny). In adversarial training, the model's "thirst" for depth is much stronger. The adversarial manifold is likely far more complex than the clean manifold, requiring significantly more parameters to characterize.
  • Convolutional Filters are Robust; Normalization is Not: The authors found that filters learned only on adversarial images can still classify clean images quite well (after a quick BN fine-tuning). This implies the core features are shared, but the way we normalize those features is the weak link.

Key Results

  • Normalization Gains: By using MBN to disentangle clean and adversarial statistics, the authors achieved strong robustness without sacrificing the ability to train on clean data.
  • Consistency Matters: Enforcing consistent BN behavior between training and testing (via the running stats heuristic) improved robustness by up to 3.0%.
  • Scaling Breakthrough: Increasing depth from ResNet-152 to ResNet-338 improved robustness by 2.4%, whereas the same jump in standard training only improved clean accuracy by 0.5%.
  • State-of-the-Art Scaling: Their ResNet-638 reached an unprecedented level of robustness on ImageNet, suggesting that we haven't yet reached the "overfitting" stage for adversarial tasks.

Limitations and Future Work

  • The Robustness-Accuracy Trade-off: As with most AT research, there is a "price" for robustness. Models that are highly resistant to attacks often show a decrease in accuracy on clean, unperturbed images.
  • Computational Cost: Training a ResNet-638 on ImageNet is computationally expensive, making this approach difficult for researchers without massive GPU clusters.
  • Domain Gap Specifics: While the "two-domain" hypothesis is supported by the data, the exact mathematical nature of the difference between the "clean domain" and "adversarial domain" remains an open question for theoretical exploration.

Summary updated

Abstract

Adversarial training is one of the main defenses against adversarial attacks. In this paper, we provide the first rigorous study on diagnosing elements of large-scale adversarial training on ImageNet, which reveals two intriguing properties. First, we study the role of normalization. Batch normalization (BN) is a crucial element for achieving state-of-the-art performance on many vision tasks, but we show it may prevent networks from obtaining strong robustness in adversarial training. One unexpected observation is that, for models trained with BN, simply removing clean images from training data largely boosts adversarial robustness, i.e., 18.3%. We relate this phenomenon to the hypothesis that clean images and adversarial images are drawn from two different domains. This two-domain hypothesis may explain the issue of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics of this mixture distribution is challenging. Guided by this two-domain hypothesis, we show disentangling the mixture distribution for normalization, i.e., applying separate BNs to clean and adversarial images for statistics estimation, achieves much stronger robustness. Additionally, we find that enforcing BNs to behave consistently at training and testing can further enhance robustness. Second, we study the role of network capacity. We find our so-called "deep" networks are still shallow for the task of adversarial learning. Unlike traditional classification tasks where accuracy is only marginally improved by adding more layers to "deep" networks (e.g., ResNet-152), adversarial training exhibits a much stronger demand on deeper networks to achieve higher adversarial robustness. This robustness improvement can be observed substantially and consistently even by pushing the network capacity to an unprecedented scale, i.e., ResNet-638.
BibTeX
@inproceedings{xie2020intriguing,
  author = {Cihang Xie and Alan L. Yuille},
  title = {{Intriguing Properties of Adversarial Training at Scale}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=FP_DSpW28E},
}
Figure