What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness?

Nikolaos Tsilivis, Julia Kempe

Introduction

Deep neural networks, despite their state-of-the-art performance, remain notoriously brittle. Small, imperceptible perturbations—known as adversarial examples—can easily cause a model to misclassify an image. While researchers have hypothesized that this vulnerability stems from a reliance on "non-robust features" (patterns that are predictive but highly sensitive), we have lacked a formal mathematical framework to measure and visualize these features.

This research bridges the gap between deep learning theory and adversarial robustness by leveraging the Neural Tangent Kernel (NTK). The NTK describes how neural networks behave during training; in the "lazy regime" (associated with very wide networks), the NTK remains nearly constant, allowing for a linearized analysis of the model's behavior. By studying the spectral properties of the NTK, we can dissect the internal features of a network and categorize them by their robustness and utility.

Methods: The NTK Framework

The authors utilize the NTK framework to study robustness through three primary lenses:

  1. Analytical Attack Generation: In the lazy regime, the network's output is approximately a linear function of its parameters governed by the NTK. This allows for the derivation of "training-free" adversarial attacks. These perturbations are calculated using only the model architecture and training data, without needing the final trained weights.
  2. Spectral Anatomy of Features: By performing an eigendecomposition of the NTK, the authors decompose the network's predictions into a sum of individual "features" (eigenvectors). These features are then evaluated based on their usefulness (contribution to accuracy) and robustness (resistance to noise).
  3. Kernel Dynamics: The study tracks how the empirical NTK of finite-width networks changes during adversarial training (e.g., using Projected Gradient Descent) compared to standard training.

Key Insights: Robustness and the Spectrum

Adversarial Examples as Architectural Blind Spots

The authors demonstrate that adversarial attacks derived purely from the NTK successfully transfer to fool real, finite-width neural networks. In some cases, these analytical attacks can drive the robust accuracy of a trained wide network down to 0%. This suggests that vulnerability is often inherent to the architecture and data initialization, rather than being a unique byproduct of the specific weights learned during training.

The Robustness-Accuracy Trade-off

The eigendecomposition reveals a clear hierarchy in how networks perceive data:

  • Robust Features: These correspond to the largest eigenvalues of the NTK. They represent global, stable patterns and are typically the first features the network learns.
  • Useful but Non-Robust Features: These reside in the "tail" of the spectrum (small eigenvalues). While they are highly predictive and help the model achieve high standard accuracy, they are easily destroyed by tiny perturbations.

Standard training processes often rely on these tail features to squeeze out extra performance, which inadvertently introduces brittleness. The NTK framework allows for the visualization of these features; they often appear as high-frequency noise to humans but are mathematically predictive for the model.

Adversarial Training as a Spectral Filter

The research sheds light on why adversarial training (AT) works. By quantifying the evolution of the empirical NTK, the authors found that AT causes the network to enter the "lazy" (linearized) regime much earlier in the training process than standard training. Effectively, AT acts as a filter that "muffles" non-robust features and forces the network to prioritize the top, robust eigenspaces. This manifests as a much stronger bias toward learning features within the most stable components of the kernel.

Conclusion and Future Work

By applying the NTK framework to adversarial robustness, this work provides a unified perspective on why neural networks fail and how they can be strengthened. The discovery that robust features align with the top eigenspaces of the kernel offers a path toward designing architectures with inherent stability.

While the computational complexity of computing the NTK eigendecomposition currently limits this analysis to benchmarks like MNIST and CIFAR-10, the insights provide a rigorous foundation. Future work may explore how these spectral properties hold up in the "rich" (feature-learning) regime and whether more complex architectures, such as Transformers, exhibit similar spectral robustness properties.

Summary updated

Abstract

The adversarial vulnerability of neural nets, and subsequent techniques to create robust models have attracted significant attention; yet we still lack a full understanding of this phenomenon. Here, we study adversarial examples of trained neural networks through analytical tools afforded by recent theory advances connecting neural networks and kernel methods, namely the Neural Tangent Kernel (NTK), following a growing body of work that leverages the NTK approximation to successfully analyze important deep learning phenomena and design algorithms for new applications. We show how NTKs allow to generate adversarial examples in a ``training-free'' fashion, and demonstrate that they transfer to fool their finite-width neural net counterparts in the ``lazy'' regime. We leverage this connection to provide an alternative view on robust and non-robust features, which have been suggested to underlie the adversarial brittleness of neural nets. Specifically, we define and study features induced by the eigendecomposition of the kernel to better understand the role of robust and non-robust features, the reliance on both for standard classification and the robustness-accuracy trade-off. We find that such features are surprisingly consistent across architectures, and that robust features tend to correspond to the largest eigenvalues of the model, and thus are learned early during training. Our framework allows us to identify and visualize non-robust yet useful features. Finally, we shed light on the robustness mechanism underlying adversarial training of neural nets used in practice: quantifying the evolution of the associated empirical NTK, we demonstrate that its dynamics falls much earlier into the ``lazy'' regime and manifests a much stronger form of the well known bias to prioritize learning features within the top eigenspaces of the kernel, compared to standard training.
BibTeX
@inproceedings{tsilivis2022what,
  author = {Nikolaos Tsilivis and Julia Kempe},
  title = {{What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness?}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=KBUgVv8z7OA},
}

Loading related content...

Figure