Rethinking Adversarial Training with Neural Tangent Kernel

Guanlin Li, Han Qiu, Shangwei Guo, Jiwei Li, Tianwei Zhang

Abstract

Adversarial training (AT) is an important and attractive topic in deep learning security, exhibiting mysteries and odd properties. Recent studies of neural network training dynamics based on Neural Tangent Kernel (NTK) make it possible to reacquaint AT and deeply analyze its properties. In this paper, we perform an in-depth investigation of AT process and properties with NTK, such as NTK evolution. We uncover three new findings that are missed in previous works. First, we disclose the impact of data normalization on AT and the importance of unbiased estimators in batch normalization layers. Second, we experimentally explore the kernel dynamics and propose more time-saving AT methods. Third, we study the spectrum feature inside the kernel to address the catastrophic overfitting problem. To the best of our knowledge, it is the first work leveraging the observations of kernel dynamics to improve existing AT methods.

Introduction

Adversarial Training (AT) remains the most effective defense against adversarial perturbations, yet it is often viewed as a "black box" with several drawbacks: high computational overhead, degradation of clean accuracy, and susceptibility to catastrophic overfitting. This study utilizes the Neural Tangent Kernel (NTK) framework to analyze the training dynamics of models under adversarial conditions, providing a more transparent view of how these models learn and specialize.

The authors employ the traced NTK to monitor training via three key metrics:

  1. Kernel Distance (KD): Quantifies the change in the kernel between epochs.
  2. Kernel Effective Rank (KER): Indicates the complexity and flatness of the feature space.
  3. Kernel Specialization (KS): Measures the alignment of the kernel with specific class labels.

By analyzing these metrics on both clean data (C-NTKs) and adversarial examples (AE-NTKs), the study identifies critical discrepancies in model perception and learning phases.

The Three-Stage Evolution of Kernel Dynamics

A central finding of this research is that AT does not progress linearly. Instead, it follows a distinct three-stage evolution:

  1. Initial Kernel Learning Phase: Occurs during the first ~10 epochs, where the kernel rapidly adapts.
  2. Lazy Training Phase: A long middle period where the kernel remains relatively static, despite continued training.
  3. Final Kernel Learning Burst: Triggered by the final learning rate decay, where the kernel undergoes another significant shift and specialization.
Evolution of KD, KER, and KS metrics during training, revealing a three-stage process.

This observation suggests that the middle phase of training is computationally inefficient, as the model's fundamental feature representation (the kernel) is not significantly changing.

The Critical Role of Batch Normalization Buffers

The study highlights a previously overlooked aspect of model robustness: the "running" statistics (mean and variance) stored in Batch Normalization (BN) buffers. While these buffers have a negligible impact on clean accuracy, they are shown to be essential for robust accuracy.

Impact of Batch Normalization buffers on clean and robust accuracy across different training regimes.

Empirical results demonstrate that removing these buffers prevents the model from effectively specializing its kernel for adversarial examples, leading to a failure in robustness. This underscores the importance of unbiased estimators in BN layers for successful AT.

Practical Implications: Efficiency and Stability

Leveraging the "lazy" middle phase of training, the authors propose a time-saving AT paradigm. By training on clean data for the first 140 epochs and switching to expensive multi-step AT only for the final 60 epochs, they achieved a 50.56% reduction in training time while maintaining robustness levels comparable to full AT (e.g., 52.82% vs. 55.17% PGD-20 accuracy on CIFAR-10).

Furthermore, the research addresses catastrophic overfitting in single-step AT (like FGSM). NTK analysis reveals that models often learn "shortcut" features—isotropic noise patterns that appear robust during training but fail to generalize. By introducing "anisotropy noise" during data augmentation, the authors successfully broke these shortcuts, stabilizing single-step AT and preventing the sudden collapse of robust accuracy.

Conclusion and Future Work

This work provides a novel perspective on Adversarial Training through the lens of kernel dynamics. By identifying the three-stage evolution of the NTK and the critical role of BN buffers, the authors offer practical strategies to make AT more efficient and stable. Future research will aim to provide a deeper theoretical explanation for the kernel's "re-awakening" after learning rate decay and extend these insights to mitigate standard overfitting in multi-step adversarial training.

Summary updated

Abstract

Adversarial training (AT) is an important and attractive topic in deep learning security, exhibiting mysteries and odd properties. Recent studies of neural network training dynamics based on Neural Tangent Kernel (NTK) make it possible to reacquaint AT and deeply analyze its properties. In this paper, we perform an in-depth investigation of AT process and properties with NTK, such as NTK evolution. We uncover three new findings that are missed in previous works. First, we disclose the impact of data normalization on AT and the importance of unbiased estimators in batch normalization layers. Second, we experimentally explore the kernel dynamics and propose more time-saving AT methods. Third, we study the spectrum feature inside the kernel to address the catastrophic overfitting problem. To the best of our knowledge, it is the first work leveraging the observations of kernel dynamics to improve existing AT methods.
BibTeX
@misc{li2023rethinking,
  author = {Guanlin Li and Han Qiu and Shangwei Guo and Jiwei Li and Tianwei Zhang},
  title = {{Rethinking Adversarial Training with Neural Tangent Kernel}},
  year = {2023},
  eprint = {2312.02236},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.02236},
}
Figure