Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"

Roland S. ZimmermannarXiv 2019 · OpenReview

TL;DR

This comment argues that a previous paper's claim of improved adversarial robustness in Bayesian Neural Networks (BNNs) might be an artifact of an insufficient attack method. By proposing a more appropriate attack that accounts for the inherent stochasticity of BNNs, the author demonstrates that the claimed robustness benefits are significantly reduced, highlighting the critical need for tailored attacks when evaluating defenses against probabilistic models.

What Problem Does This Paper Address?

The paper addresses the challenge of accurately evaluating the robustness of neural networks against adversarial attacks, particularly when dealing with more complex model architectures like Bayesian Neural Networks (BNNs). Adversarial training is a common defense strategy, but its effectiveness hinges on the quality of the adversarial examples used during training and evaluation. The original "Adv-BNN" paper claimed that adversarially training BNNs led to superior robustness compared to standard, non-Bayesian networks. However, this comment questions whether this observed robustness is a genuine improvement or an artifact of the attack method used. Specifically, it points out that standard adversarial attack algorithms might not be well-suited for BNNs due to their inherent stochasticity, potentially leading to an overestimation of their defense capabilities.

Methods

The core of this comment lies in proposing a modified adversarial attack strategy. The original "Adv-BNN" paper used a "naive PGD" attack that sampled a new random vector (representing the stochasticity of the BNN) at each update step. This comment argues that this approach can lead to unstable gradient estimates. To address this, it introduces an "averaged PGD" (A-PGD) attack. This new method estimates the true gradient of the network by averaging the gradients over multiple random vectors sampled from the BNN's distribution. This averaging aims to provide a more stable and representative gradient direction, akin to the Expectation Over Transformation (EOT) attack. The author then uses this A-PGD attack to re-evaluate the robustness of the Adv-BNN and other defense methods.

Datasets

The experiments and evaluations in this paper were conducted on the STL-10 dataset. The author notes that experiments on CIFAR10 and ImageNet were not performed due to issues with the published implementation of the original Adv-BNN.

Key Insights and Ideas

  • The importance of tailored attacks: Evaluating the robustness of probabilistic models like BNNs requires adversarial attacks that properly account for their stochastic nature. Standard attacks might not be sufficient.
  • Stochasticity can mask true robustness: The "naive PGD" attack used in the original Adv-BNN paper might have underestimated the true vulnerability of the BNNs by not accurately capturing the average gradient behavior.
  • Averaging gradients is key: The proposed A-PGD attack, by averaging gradients over multiple stochastic samples, provides a more stable and representative gradient for generating adversarial examples against BNNs.
  • Re-evaluation challenges claims: When a more appropriate attack (A-PGD) is used, the claimed significant robustness advantage of Adv-BNN over standard adversarially trained networks diminishes.

Key Results

The author reproduced the results of the original Adv-BNN paper using the "naive PGD" attack on the STL-10 dataset, observing the reported accuracy improvements. However, when the same defended networks were tested using the proposed "averaged PGD" (A-PGD) attack:

  • The accuracy of the Adv-BNN dropped significantly compared to the naive PGD evaluation.
  • An adversarially trained standard CNN (Adv. Training) showed better robustness than the Adv-BNN when attacked with A-PGD, especially for larger perturbation bounds.
  • When the Adv-BNN was adversarially trained using the A-PGD attack itself ("Adv-BNN w/ A-PGD"), its robustness improved, yielding slightly higher accuracy than the other methods, though the author suggests this advantage might be marginal and dependent on hyperparameter choices.

For example, with a perturbation bound of $\gamma = 0.07$ and using the A-PGD attack:

  • Adv. Training: 8.6% accuracy
  • Adv-BNN: 21.1% accuracy
  • Adv-BNN w/ A-PGD: 13.4% accuracy

Limitations and Future Work

  • Limited dataset scope: The analysis is primarily limited to the STL-10 dataset due to implementation issues with other datasets.
  • Hyperparameter sensitivity: The author notes that the improved performance of "Adv-BNN w/ A-PGD" might be sensitive to hyperparameter choices.
  • Future work implied: The paper strongly suggests that future research on defending BNNs must incorporate more sophisticated attack methods that account for their probabilistic nature. The development and application of such tailored attacks are crucial for a fair and accurate assessment of their robustness.

Summary updated

Abstract

A recent paper by Liu et al. combines the topics of adversarial training and Bayesian Neural Networks (BNN) and suggests that adversarially trained BNNs are more robust against adversarial attacks than their non-Bayesian counterparts. Here, I analyze the proposed defense and suggest that one needs to adjust the adversarial attack to incorporate the stochastic nature of a Bayesian network to perform an accurate evaluation of its robustness. Using this new type of attack I show that there appears to be no strong evidence for higher robustness of the adversarially trained BNNs.
BibTeX
@misc{zimmermann2019comment,
  author = {Roland S. Zimmermann},
  title = {{Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"}},
  year = {2019},
  eprint = {1907.00895},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1907.00895},
}
Figure