Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach

Shaopeng Fu, Di Wang

Introduction

Adversarial training (AT) is widely regarded as the most effective defense against adversarial attacks, which involve small, human-imperceptible perturbations designed to mislead deep neural networks (DNNs). Despite its success, AT exhibits a counterintuitive behavior known as robust overfitting. Unlike standard training, where longer training typically improves or stabilizes performance, prolonged AT often leads to a significant decline in a model's robustness against new adversarial examples, even as training loss continues to decrease.

While empirical evidence of robust overfitting is abundant, a rigorous theoretical explanation has remained elusive, particularly for deep and wide architectures. This paper bridges that gap by extending Neural Tangent Kernel (NTK) theory to the adversarial setting, providing a mathematical framework to explain why wide DNNs lose their robustness over time.

Theoretical Framework: Extending NTK to Adversarial Training

The Neural Tangent Kernel (NTK) is a powerful tool used to analyze the training dynamics of infinitely wide neural networks. Under standard training, a wide DNN behaves like a linear model evolving under gradient descent. To apply this to adversarial training, the authors introduce several non-trivial extensions:

  1. Continuous Gradient Flow for Attacks: Traditional AT relies on discrete optimization (like Projected Gradient Descent) to find adversarial perturbations. To make the analysis mathematically tractable, the authors model the adversarial attack as a continuous gradient flow, replacing hard constraints with a learning rate parameter that dictates the "strength" of the adversarial search.
  2. Linearization of AT Dynamics: The authors prove that an adversarially trained wide DNN can be strictly approximated by its linearization—a Taylor expansion around its initial parameters. This allows the complex evolution of a deep network during AT to be studied through a simpler, linear lens.
  3. Closed-Form Solutions: By focusing on the squared loss function, the researchers derived closed-form expressions for the training dynamics. This mathematical breakthrough reveals that AT introduces a time-dependent regularization matrix, denoted as $\Xi(t)$, which modifies the standard NTK equations.

The AT Degeneration Phenomenon

The most significant insight from this theoretical analysis is the discovery of AT Degeneration. The study shows that the robustness gained during training is tied to the regularization matrix $\Xi(t)$. However, this effect is transient.

As training time $t$ approaches infinity, the influence of the regularization matrix $\Xi(t)$ eventually vanishes. Mathematically, the model's state converges to the same solution it would have reached under standard (clean) training. This "forgetting" of robust features explains why robust accuracy peaks early and then declines—a theoretical confirmation of the necessity of early stopping in adversarial training.

Adv-NTK: Robustness for Infinite-Width Networks

Building on their theoretical derivations, the authors propose Adv-NTK, the first algorithm designed to train robust infinite-width neural networks. By incorporating the adversarial regularization terms directly into the NTK framework, Adv-NTK allows researchers to study the limits of robustness in the infinite-width regime.

Experimental Results

The authors validated their findings through experiments on the CIFAR-10 and SVHN datasets using various architectures, including Multi-Layer Perceptrons (MLPs) and ResNets (modified to remove Global Average Pooling for computational feasibility).

  • Performance Gains: On an 8-layer MLP using CIFAR-10, Adv-NTK achieved a robust accuracy of 30.91%, a significant improvement over the 24.43% achieved by standard NTK.
  • Consistency with Theory: The robust accuracy curves of the linearized models closely mirrored the "increase-then-decrease" patterns observed in finite-width networks. This alignment suggests that the linearized NTK model captures the essential dynamics of robust overfitting.
  • Infinite vs. Finite Width: While Adv-NTK represents a leap forward for infinite-width models, the authors noted a "non-linearity gap." Finite-width networks at their best checkpoints still tend to outperform the linearized Adv-NTK, suggesting that the inherent non-linearities of finite networks contribute additional robust features.

Conclusion and Limitations

This research provides a foundational theoretical explanation for robust overfitting in wide DNNs. By demonstrating that adversarial regularization naturally decays over time, the authors justify the empirical practice of early stopping and provide a new tool, Adv-NTK, for exploring robust kernels.

However, several challenges remain. The current closed-form dynamics are specific to the squared loss function, whereas most practical applications use cross-entropy loss. Additionally, the computational cost of NTK methods currently limits their application to architectures without Global Average Pooling. Future work will likely focus on bridging these gaps and finding ways to stabilize the adversarial regularization term to prevent the degeneration of robustness in long-term training.

Summary updated

Abstract

Adversarial training (AT) is a canonical method for enhancing the robustness of deep neural networks (DNNs). However, recent studies empirically demonstrated that it suffers from robust overfitting, i.e., a long time AT can be detrimental to the robustness of DNNs. This paper presents a theoretical explanation of robust overfitting for DNNs. Specifically, we non-trivially extend the neural tangent kernel (NTK) theory to AT and prove that an adversarially trained wide DNN can be well approximated by a linearized DNN. Moreover, for squared loss, closed-form AT dynamics for the linearized DNN can be derived, which reveals a new AT degeneration phenomenon: a long-term AT will result in a wide DNN degenerates to that obtained without AT and thus cause robust overfitting. Based on our theoretical results, we further design a method namely Adv-NTK, the first AT algorithm for infinite-width DNNs. Experiments on real-world datasets show that Adv-NTK can help infinite-width DNNs enhance comparable robustness to that of their finite-width counterparts, which in turn justifies our theoretical findings. The code is available at https://github.com/fshp971/adv-ntk.
BibTeX
@inproceedings{fu2024theoretical,
  author = {Shaopeng Fu and Di Wang},
  title = {{Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=1op5YGZu8X},
}
Figure