Not So Robust after All: Evaluating the Robustness of Deep Neural Networks to Unseen Adversarial Attacks

Roman Garaev, Bader Rasheed, Adil Mehmood KhanAlgorithms 2024 · DOI

TL;DR

This paper investigates the robustness of deep neural networks (DNNs) against adversarial attacks, particularly focusing on the transferability of defense mechanisms like robust and adversarial training. The authors reveal a significant lack of generalization for these defenses against unseen attacks, especially those employing L∞-norm perturbations, suggesting current evaluation methods might provide a false sense of security.

What Problem Does This Paper Address?

Deep neural networks (DNNs) have achieved remarkable success in various applications, but their susceptibility to adversarial attacks remains a critical concern. Adversarial attacks involve making subtle, often imperceptible, changes to input data that can cause DNNs to misclassify them. While defense mechanisms like robust training (training on "clean" features) and adversarial training (training on adversarial examples) have been proposed, their effectiveness and generalizability against a wide range of attacks are not fully understood. This paper aims to bridge this gap by rigorously evaluating these defenses against various unseen adversarial attacks, with a particular focus on the often-underestimated impact of L∞-norm perturbations.

Methods

The authors employ a multi-faceted approach to evaluate DNN robustness:

  • Expanded Adversarial Training and Robust Training Experiments: They replicate and extend previous work on robust and adversarial training. This involves training models on modified datasets (robust training) or incorporating adversarial examples into the training process (adversarial training).
  • Broad Testing Setup: To assess generalizability, they test these trained models against a variety of unseen attacks, including FGSM, PGD, C-W, and DeepFool, using different perturbation norms (L1, L2, and L∞) and epsilon values.
  • Canonical Correlation Analysis (CCA): Singular Value Canonical Correlation Analysis (SVCCA) is used to compare the latent representations of benign and adversarial samples. This helps understand how attacks affect the internal workings of the network and whether representations remain similar.
  • Principal Component Analysis (PCA) for Visualization: PCA is employed to reduce the dimensionality of the high-dimensional representation vectors, allowing for visual analysis of how different classes' representations intermingle under various attacks.
  • Decision Boundary Analysis: The authors investigate the impact of adversarial training on the network's decision boundary by measuring the distance between samples and the boundary using the DeepFool attack.

Datasets

The paper primarily uses the following datasets:

  • CIFAR-10: A widely used dataset for image classification, employed for training and testing models.
  • CINIC-10: Another image classification dataset, used for broader testing and validation.

The authors do not introduce new datasets but rather use existing ones to conduct their extensive experiments.

Key Insights and Ideas

  • L∞-norm Attacks are More Potent Than Often Assumed: The study strongly suggests that L∞-norm attacks are significantly more effective at compromising DNNs than L2-norm attacks, even when the perturbation magnitudes appear similar to humans. This challenges the common practice of evaluating robustness primarily against L2-norm attacks.
  • Defense Mechanisms Lack Generalization: Both robust and adversarial training methods, while effective against the specific attacks they were trained on, show a significant drop in performance when faced with unseen attacks, particularly those with L∞-norm perturbations. This indicates that these defenses do not guarantee broad robustness.
  • Non-Robust Features are Still Exploitable: The hypothesis that adversarial attacks solely exploit non-robust features is challenged. The experiments suggest that even models trained on "robust" features can be compromised, implying a more complex interplay of features and attack strategies.
  • Impact on Latent Representations: L∞-norm attacks cause greater dispersion in the latent representations of DNNs compared to L2-norm attacks. This mixing of representations makes classification more difficult.
  • Decision Boundary Proximity: Adversarial training, especially with L∞-norm attacks, can paradoxically decrease the distance between samples and the decision boundary, making the model easier to misclassify with smaller perturbations.

Key Results

  • Accuracy Degradation: Tables and figures throughout the paper demonstrate substantial accuracy drops for adversarially and robustly trained models when subjected to unseen L∞-norm attacks. For instance, in Table 3, an adversarially trained ResNet18 model shows a significant decrease in accuracy against L∞-norm PGD attacks compared to L2-norm attacks.
  • SVCCA Findings: Mean correlation coefficients from SVCCA (Table 7) show that L∞-norm perturbations have a more significant impact on network representations, even for adversarially trained models, indicating a potential "false sense of security" from L2-norm testing.
  • PCA Visualizations: Figures 4-8 visually confirm that adversarial attacks, especially L∞-norm, cause representations of different classes to become heavily mixed, making classification challenging. Adversarial training helps to cluster representations but doesn't fully resolve the mixing issue for L∞-norm attacks.
  • Decision Boundary Distances: Table 8 shows that for some adversarially trained models, the mean distance to the decision boundary is smaller, particularly for L∞-norm training, suggesting increased vulnerability.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that the choice of epsilon values in the literature for adversarial attacks might be too loose, contributing to the observed vulnerabilities.
  • Computational constraints limited the scope of some experiments, such as the use of ResNet18 for PCA visualizations.
  • The paper focuses on white-box scenarios where the adversary has full knowledge of the model.
  • Future Work:
    • The authors suggest further research into determining a reasonable range of perturbation coefficients for robust models.
  • They recommend that researchers pay closer attention to L∞-norm attacks to avoid a false sense of security and ensure more comprehensive model robustness testing.
  • Investigating the reliability of current model testing methodologies is also highlighted as an area for future exploration.

Summary updated

BibTeX
@article{garaev2024not,
  author = {Roman Garaev and Bader Rasheed and Adil Mehmood Khan},
  title = {{Not So Robust after All: Evaluating the Robustness of Deep Neural Networks to Unseen Adversarial Attacks}},
  journal = {Algorithms},
  year = {2024},
  doi = {10.3390/a17040162},
}
Figure