Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, Ananthram Swami

Abstract

Deep Neural Networks (DNNs) are vulnerable to adversarial perturbations—small, human-imperceptible changes to input data that cause misclassification. This paper proposes defensive distillation, a training technique that leverages the distillation process to reduce the sensitivity of DNNs to such perturbations. By training a student model on "soft" labels generated by a high-temperature teacher model, the defense effectively smooths the model's gradients. Experimental results on MNIST and CIFAR-10 show that defensive distillation reduces the success rate of adversarial attacks from over 87% to under 5% while maintaining high classification accuracy.

Introduction

Deep learning models, despite their state-of-the-art performance, are notoriously brittle. Researchers have demonstrated that adding tiny, human-imperceptible "noise" to an image can trick a model into misclassifying a car as a cat or a stop sign as a yield sign. This vulnerability stems from the extreme sensitivity of DNNs; they often possess very steep gradients. An attacker can exploit these gradients to identify the precise direction in which a minimal change to input pixels will cause a significant shift in the output classification.

Previous defensive measures often required architectural changes or offered only partial protection. This work introduces a defense that is easy to implement, preserves model accuracy, and addresses the root cause: the model's sensitivity to input variations.

Methodology: Defensive Distillation

The authors adapt the concept of Distillation, originally designed for model compression, into a defensive strategy. The core idea is to use the knowledge of a "teacher" model to train a "student" model of the same architecture in a way that regularizes the mapping from inputs to outputs.

Architecture of a Deep Neural Network with a Softmax output layer used in the distillation process.

The defensive distillation process follows four primary steps:

  1. Teacher Training: A standard DNN ($F$) is trained on the original dataset using a modified Softmax layer with a Temperature ($T$) parameter. A high $T$ (e.g., $T=20$) "smooths" the output probability distribution, revealing relative relationships between classes (e.g., showing that a "7" shares features with a "1").
  2. Soft Label Generation: The trained teacher generates "soft labels" for the training data. Instead of binary "hard labels," these provide a probability distribution (e.g., 60% probability for "7", 38% for "1").
  3. Student Training: A second DNN ($F^d$) with the exact same architecture is trained using these soft labels at the same high temperature $T$.
  4. Inference: During testing, the temperature is reset to $T=1$.

Training on soft labels at high temperatures acts as a powerful regularizer, forcing the model to learn a smoother mapping and flattening the gradients that attackers rely on.

Experimental Evaluation

The defense was evaluated on two benchmark computer vision datasets:

  • MNIST: 70,000 grayscale images of handwritten digits (0-9).
  • CIFAR-10: 60,000 color images across 10 classes (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck).

Key Results and Insights

The primary achievement of defensive distillation is the dramatic reduction in gradient magnitude. The authors found that the defense reduces the magnitude of gradients used by attackers by a factor of $10^{30}$, effectively "blinding" gradient-based attack algorithms.

Adversarial examples from MNIST and CIFAR-10 datasets showing original images and their perturbed counterparts that cause misclassification.

Attack Success Rates

The impact on robustness is significant:

  • MNIST: The success rate for crafting adversarial samples dropped from 95.89% to 0.45%.
  • CIFAR-10: The success rate dropped from 87.89% to 5.11%.

Furthermore, the defense increased the average minimum number of pixels an attacker must modify to achieve misclassification by 800% on MNIST.

Robustness vs. Accuracy

Crucially, defensive distillation maintains high performance on clean data. Unlike other defenses that significantly degrade accuracy, this method resulted in a difference of less than 1.4% compared to undefended models. The temperature $T$ serves as a critical hyperparameter; as $T$ increases, robustness improves until reaching a plateau (around $T=40$ for CIFAR-10).

Conclusion

Defensive distillation provides a robust mechanism for hardening DNNs against adversarial perturbations by smoothing the model's decision surface. While highly effective against current gradient-based attacks, the authors acknowledge the ongoing "arms race" in adversarial machine learning. Future work includes extending this technique to non-Softmax architectures and exploring its applicability to tasks beyond image classification.

Summary updated

Abstract

Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 1030. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.
BibTeX
@inproceedings{papernot2016distillation,
  author = {Nicolas Papernot and Patrick D. McDaniel and Xi Wu and Somesh Jha and Ananthram Swami},
  title = {{Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks}},
  booktitle = {IEEE Symposium on Security and Privacy},
  year = {2016},
  doi = {10.1109/sp.2016.41},
}
Figure