Understanding Robustness in Teacher-Student Setting: A New Perspective

Zhuolin Yang, Zhaoxi Chen, Tiffany Cai, Xinyun Chen, Bo Li, Yuandong Tian

Introduction

Deep learning models are notoriously fragile to adversarial examples—tiny, often invisible perturbations that cause confident, incorrect predictions. While theories like the "non-robust features" hypothesis exist, the internal mechanisms within hidden layers that lead to this vulnerability remain opaque. This paper introduces a novel "teacher-student" framework to demystify this phenomenon, demonstrating that adversarial vulnerability arises when student neurons fail to "specialize" or align with teacher neurons outside the narrow subspace of training data.

The Teacher-Student Framework

To analyze the internal representations of neural networks, the researchers utilize a teacher-student setting. In this paradigm, a "teacher" network ($f^*$) acts as an oracle providing ground-truth labels for any possible input, while a "student" network ($f$) is trained to mimic it. This allows for a direct comparison between the internal logic of a robust oracle and a potentially vulnerable learner.

Figure 1: Conceptual overview of the teacher-student framework and neuron specialization within data subspaces.

The core theory posits that in "low-rank" settings—where data like real-world images lives on a lower-dimensional manifold—student neurons naturally align with teacher neurons within that specific data subspace. However, without explicit pressure, student neurons can diverge significantly from the teacher in directions orthogonal to the data subspace. The researchers decompose the student's weight vector into "in-plane" (on the data manifold) and "out-of-plane" (off the manifold) error components to track this divergence.

Specialization and Adversarial Vulnerability

The "aha moment" of this research is identifying adversarial examples as "unspecialized" gaps. These examples live in regions where student neurons have not yet aligned with the teacher's logic. If a student neuron hasn't specialized to a teacher neuron in a certain direction, an attacker can exploit that direction to manipulate the model's output.

To measure this specialization, the authors use Normalized Correlation (NC), tracking how closely a student neuron's activations correlate with a teacher neuron's activations across a dataset. They found a strong negative correlation (Pearson score ~ -0.6) between specialization error ($\epsilon_{in}$) and NC, confirming that higher specialization consistently leads to higher robust accuracy.

Results: Why Adversarial Training Works

The study reveals that Adversarial Training (AT) acts as a targeted form of data augmentation. By training on adversarial examples, the model is forced to encounter data in its "weak" directions, pushing student neurons to specialize even in the out-of-plane regions where they were previously chaotic.

Figure 5: Sorted Best Normalized Correlation (BNC) curves across different layers, comparing standard training (ST) with adversarial training (AT) over multiple epochs.

As shown in the visualization of Sorted BNC Curves, AT forces almost all neurons in the network to align with a corresponding teacher neuron. In contrast, standard training (ST) leaves many neurons "unspecialized," rendering them effectively useless or even harmful for robustness. Notably, an AT model trained for only 50 epochs achieves higher Mean Best Normalized Correlation (MBNC) than a standard model trained for 300 epochs.

Conclusion and Future Work

This framework provides a clear lens through which to view adversarial robustness: it is a matter of internal alignment. While the study focused on relatively shallow CNNs (4-5 layers) on CIFAR-10, the insights suggest that robustness can be improved by forcing specialization.

Future work may explore whether these findings hold for larger architectures like Vision Transformers. Additionally, the authors suggest developing "label-extrapolation" techniques to teach students about the space surrounding the data manifold without the heavy computational cost of iterative adversarial training.

Summary updated

Abstract

Adversarial examples have appeared as a ubiquitous property of machine learning models where bounded adversarial perturbation could mislead the models to make arbitrarily incorrect predictions. Such examples provide a way to assess the robustness of machine learning models as well as a proxy for understanding the model training process. Extensive studies try to explain the existence of adversarial examples and provide ways to improve model robustness (e.g. adversarial training). While they mostly focus on models trained on datasets with predefined labels, we leverage the teacher-student framework and assume a teacher model, or oracle, to provide the labels for given instances. We extend Tian (2019) in the case of low-rank input data and show that student specialization (trained student neuron is highly correlated with certain teacher neuron at the same layer) still happens within the input subspace, but the teacher and student nodes could differ wildly out of the data subspace, which we conjecture leads to adversarial examples. Extensive experiments show that student specialization correlates strongly with model robustness in different scenarios, including student trained via standard training, adversarial training, confidence-calibrated adversarial training, and training with robust feature dataset. Our studies could shed light on the future exploration about adversarial examples, and enhancing model robustness via principled data augmentation.
BibTeX
@inproceedings{yang2021understanding,
  author = {Zhuolin Yang and Zhaoxi Chen and Tiffany Cai and Xinyun Chen and Bo Li and Yuandong Tian},
  title = {{Understanding Robustness in Teacher-Student Setting: A New Perspective}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2021},
  url = {https://openreview.net/forum?id=8cTxpuWy9_},
}
Figure