Adversarial Examples in Deep Learning: Characterization and Divergence

Wenqi Wei, Ling Liu, Stacey Truex, Lei Yu, Mehmet Emre GursoyarXiv 2018 · OpenReview

Abstract

The burgeoning success of deep learning has raised security and privacy concerns as more tasks involve sensitive data. Adversarial attacks have emerged as a dominant threat to mission-critical systems. This paper takes a holistic and principled approach to perform statistical characterization of adversarial examples. We provide a general formulation of adversarial examples and elaborate on the basic principles for attack algorithm design. We introduce "easy" and "hard" categorizations to analyze effectiveness in terms of success rate, perturbation degree, and prediction entropy. Extensive experiments across different hyperparameters and frameworks (TensorFlow, PyTorch, etc.) reveal that adversarial behavior diverges significantly due to variations in learned features. Our findings provide enlightenment on mitigation strategies towards effective countermeasures.

Introduction: The Vulnerability of Deep Neural Networks

Deep Neural Networks (DNNs) are notoriously vulnerable to adversarial examples—inputs crafted with human-imperceptible perturbations to fool models into misclassification. While many researchers focus on inventing new attacks, there is a lack of systematic understanding regarding their statistical nature. This paper addresses "blind spots" in our understanding: Why does an attack succeed on one model but fail on another with slightly different settings? How do training epochs, feature map sizes, and software frameworks affect vulnerability? Understanding this "divergence" is critical for moving from reactive defense to proactive, principled strategies.

Methodology and Characterization Metrics

The authors utilize a unified optimization framework for adversarial attacks, balancing noise injection against misclassification goals. They categorize attacks into "Easy" and "Hard" based on Success Rate (SR) and employ three primary statistical metrics:

  1. Success Rate (SR): The percentage of adversarial inputs that successfully fool the model.
  2. Degree of Change (DoC): The $L_p$ distance between original and adversarial inputs.
  3. Information Entropy: A measure of uncertainty in the model's prediction vector.

The study primarily uses the MNIST dataset to perform a deep dive into class-to-class vulnerability. Attacks like the Fast Gradient Sign Method (FGSM) and Jacobian-based Saliency Map Attack (JSMA) are tested across various hyperparameters and four major frameworks (TensorFlow, Caffe, Theano, and Torch).

Key Insights: Entropy and Divergence

A significant finding is that successful adversarial examples often result in higher prediction entropy, indicating increased model uncertainty. This suggests that monitoring the "confidence distribution" could be a viable path for detection.

Impact of perturbation magnitude (θ) on attack success and prediction entropy. Successful attacks often correlate with higher entropy in the prediction vector.

Furthermore, the research highlights "Framework Divergence." The same architecture trained on identical data behaves differently across software frameworks. TensorFlow and Theano models often showed higher vulnerability than Caffe and Torch, likely due to differences in gradient handling and internal optimizations. Hyperparameter sensitivity also plays a role; a model's vulnerabilities shift as it moves from underfitting to overfitting during training.

Key Results: Class Vulnerability and Attack Efficiency

Experimental results demonstrate that vulnerability is highly asymmetrical across classes. For instance, in FGSM tests with a perturbation magnitude of θ=0.2, digit '1' was the most vulnerable with a Success Rate of 0.995, whereas digit '2' was the most robust at 0.771.

Success Rate (SR) across MNIST digits for different perturbation magnitudes (θ). Vulnerability varies significantly by class.

While multi-step attacks like JSMA are computationally expensive (~3.7s vs 0.53s for FGSM), they achieve success with a much lower Degree of Change. Based on these observations, the authors propose defenses such as consensus-based mitigation (using multiple "views" of an input) and ensemble training across different frameworks to ensure that a single adversarial "blind spot" doesn't compromise the entire system.

Conclusion and Future Outlook

The primary limitation of this study is its focus on the MNIST dataset. While the statistical principles likely hold, the complexity of "Easy" vs. "Hard" attacks may change on high-dimensional datasets like ImageNet. Future work should extend this characterization to more complex computer vision and NLP tasks, and provide large-scale benchmarks for the proposed mitigation strategies against state-of-the-art adversarial training.

Summary updated

Abstract

The burgeoning success of deep learning has raised the security and privacy concerns as more and more tasks are accompanied with sensitive data. Adversarial attacks in deep learning have emerged as one of the dominating security threat to a range of mission-critical deep learning systems and applications. This paper takes a holistic and principled approach to perform statistical characterization of adversarial examples in deep learning. We provide a general formulation of adversarial examples and elaborate on the basic principle for adversarial attack algorithm design. We introduce easy and hard categorization of adversarial attacks to analyze the effectiveness of adversarial examples in terms of attack success rate, degree of change in adversarial perturbation, average entropy of prediction qualities, and fraction of adversarial examples that lead to successful attacks. We conduct extensive experimental study on adversarial behavior in easy and hard attacks under deep learning models with different hyperparameters and different deep learning frameworks. We show that the same adversarial attack behaves differently under different hyperparameters and across different frameworks due to the different features learned under different deep learning model training process. Our statistical characterization with strong empirical evidence provides a transformative enlightenment on mitigation strategies towards effective countermeasures against present and future adversarial attacks.
BibTeX
@misc{wei2018adversarial,
  author = {Wenqi Wei and Ling Liu and Stacey Truex and Lei Yu and Mehmet Emre Gursoy},
  title = {{Adversarial Examples in Deep Learning: Characterization and Divergence}},
  year = {2018},
  eprint = {1807.00051},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.00051},
}
Figure