Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric

Yujun Shi, Benben Liao, Guangyong Chen, Yun Liu, Ming-Ming Cheng, Jiashi FengarXiv 2019 · arXiv

Abstract

Deep neural networks (DNNs) often exhibit a trade-off between standard accuracy and adversarial robustness. This paper introduces a novel performance metric, Cross Category KL Divergence (CCKL), to mathematically disentangle accuracy into "robust" and "non-robust" components. By linking the non-robust component to the Fisher Information Matrix, we demonstrate that standard training inherently relies on sensitive, adversary-prone features. Our findings suggest that increasing model capacity is essential for leveraging higher-order, robust features to achieve both high accuracy and security.

Introduction

Deep neural networks are notoriously vulnerable to adversarial attacks—small, imperceptible perturbations that lead to incorrect predictions. While adversarial training offers a defense, it typically results in a significant drop in standard accuracy. This research investigates the theoretical roots of this trade-off, asking whether it is possible to achieve high performance and robustness simultaneously by understanding how models distinguish between classes.

Methodology: Disentangling Performance

To analyze adversarial vulnerability, the authors propose the Cross Category KL Divergence (CCKL) metric. CCKL measures the distance between a model's output probability distributions for different classes. Using a Taylor expansion, the metric is decomposed into two primary components:

  1. Non-Robust Component ($G_1$): A second-order term directly related to the Fisher Information Matrix. High values here indicate high local curvature and sensitivity to small input changes, leading to adversarial vulnerability.
  2. Robust Component ($G_2$): Higher-order terms that allow for class separation without increasing local sensitivity.
The model learns to shift its output distribution from a near-uniform state at initialization to a highly confident, class-specific distribution after training.

As shown in the figure above, training transforms the model's output $f(x)$ from an uninformative initial state to one that clearly identifies the ground truth $y$. The research focuses on which mathematical components ($G_1$ vs. $G_2$) drive this transformation.

Key Insights and Results

The Accuracy/Robustness Trade-off

Standard training tends to take the "path of least resistance," relying heavily on the $G_1$ component to achieve high accuracy quickly. Because $G_1$ is tied to Fisher Information, this process inherently increases the model's local sensitivity, making it vulnerable to perturbations. Adversarial training (e.g., PGD) acts as a constraint that suppresses $G_1$. However, if a model lacks sufficient capacity, it cannot effectively utilize the $G_2$ component, leading to the observed drop in standard accuracy.

Training Dynamics

Empirical observation of training metrics supports this theory. During standard training, metrics like KL divergence (related to CCKL) and accuracy rise together, often driven by the non-robust component.

Evolution of test accuracy, cross-entropy loss, and KL divergence over 80 training epochs on CIFAR-10.

The Role of Model Capacity

The robust $G_2$ component involves complex, higher-order mathematical terms. The authors found that increasing model capacity is the key to unlocking these features. In experiments on CIFAR-10:

  • VGG13 (Low capacity): Failed to achieve high standard (35.14%) or robust (17.06%) accuracy.
  • ResNet32x10 (High capacity): Successfully leveraged higher-order terms to achieve significantly better simultaneous performance, reaching 79.45% standard accuracy and 49.52% PGD accuracy.

Conclusion

This work provides a geometric and information-theoretic explanation for the adversarial trade-off. By identifying Fisher Information as a proxy for non-robustness, the authors show that adversarial training forces models to abandon easily exploitable local descriptors. Future work may involve treating DNNs as non-linear dynamic systems to further stabilize performance against adversarial perturbations.

Summary updated

Abstract

The vulnerability to slight input perturbations is a worrying yet intriguing property of deep neural networks (DNNs). Despite many previous works studying the reason behind such adversarial behavior, the relationship between the generalization performance and adversarial behavior of DNNs is still little understood. In this work, we reveal such relation by introducing a metric characterizing the generalization performance of a DNN. The metric can be disentangled into an information-theoretic non-robust component, responsible for adversarial behavior, and a robust component. Then, we show by experiments that current DNNs rely heavily on optimizing the non-robust component in achieving decent performance. We also demonstrate that current state-of-the-art adversarial training algorithms indeed try to robustify the DNNs by preventing them from using the non-robust component to distinguish samples from different categories. Also, based on our findings, we take a step forward and point out the possible direction for achieving decent standard performance and adversarial robustness simultaneously. We believe that our theory could further inspire the community to make more interesting discoveries about the relationship between standard generalization and adversarial generalization of deep learning models.
BibTeX
@misc{shi2019understanding,
  author = {Yujun Shi and Benben Liao and Guangyong Chen and Yun Liu and Ming-Ming Cheng and Jiashi Feng},
  title = {{Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric}},
  year = {2019},
  eprint = {1906.02494},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.02494},
}
Figure