Efficient and transferable adversarial examples from bayesian neural networks

Martin Gubri, Maxime Cordy, Mike Papadakis, Yves Le Traon, Koushik SenUAI 2022 · OpenReview

Abstract

An established way to improve the transferability of black-box evasion attacks is to craft the adversarial examples on an ensemble-based surrogate to increase diversity. We argue that transferability is fundamentally linked to model uncertainty. By using cyclical Stochastic Gradient Langevin Dynamics (cSGLD) to sample from a surrogate model's posterior distribution, we create adversarial examples that are significantly more effective and computationally cheaper than those generated by traditional ensemble methods.

Introduction

Adversarial attacks are highly effective in white-box settings where the target model's parameters are known. However, real-world scenarios often present a "black-box" challenge, where attackers must rely on the transferability of adversarial examples—the ability of an example crafted on a surrogate model to deceive an unknown target.

Traditional methods to enhance transferability often involve training ensembles of multiple independent models, which is computationally prohibitive. This paper re-frames transferability as a Bayesian problem: the unknown target model is viewed as a sample from a posterior distribution of models trained on the same dataset.

Methodology: Bayesian Surrogates

The core proposal is to replace single-point "deterministic" surrogates with Bayesian Neural Networks (BNNs). Instead of a single set of weights, BNNs capture a distribution of likely weights.

Comparison of Deterministic vs. Bayesian Transfer-based Black-box Attacks

The authors utilize cyclical Stochastic Gradient Langevin Dynamics (cSGLD) to efficiently sample from the posterior. This technique uses a cyclical learning rate to explore diverse modes of the loss landscape while adding Gaussian noise to gradients, allowing for the generation of diverse weight samples without the cost of training multiple independent models from scratch. During the attack phase, gradients are averaged across these samples (Bayesian Model Averaging), preventing the adversarial example from overfitting to the specific quirks of a single model.

Experimental Results

The approach was evaluated on MNIST, CIFAR-10, and ImageNet. A key metric introduced is the Transferability-Deep Ensemble Equivalent (T-DEE), which measures the effectiveness of the method relative to the number of independently trained models in a standard ensemble.

Performance and Transferability

On ImageNet, the cSGLD-based surrogate achieved success rates up to 83.2 percentage points higher than standard single-model surrogates. Crucially, the method demonstrated strong inter-architecture transferability, successfully attacking models like EffNetB0 using a ResNet-50 surrogate.

Success rates of cSGLD combined with other transferability-enhancing methods

As shown in the results, cSGLD is "plug-and-play." When combined with existing test-time techniques such as Input Diversity (DI), Skip Gradient Method, or Ghost Networks (GN), it consistently achieves the highest success rates. For instance, combining cSGLD with Input Diversity on a ResNet50 surrogate reached success rates over 93% against various target architectures under $L_2$ norms.

Efficiency

Efficiency is a major advantage. A single cSGLD surrogate often outperformed an ensemble of 15 independently trained DNNs while requiring significantly less computation. On ImageNet, the method achieved 94% success with 4.8x fewer FLOPs than traditional ensembles.

Conclusion

This research highlights that surrogate model training is a more powerful lever for adversarial transferability than previously thought. By framing the problem through the lens of Bayesian uncertainty and utilizing efficient sampling techniques like cSGLD, attackers can generate highly transferable adversarial examples at a fraction of the computational cost of traditional ensembles. Future work may explore aleatoric uncertainty and the effectiveness of these attacks against adversarially trained targets.

Summary updated

Abstract

An established way to improve the transferability of black-box evasion attacks is to craft the adversarial examples on an ensemble-based surrogate to increase diversity. We argue that transferabili...
BibTeX
@inproceedings{gubri2022efficient,
  author = {Martin Gubri and Maxime Cordy and Mike Papadakis and Yves Le Traon and Koushik Sen},
  title = {{Efficient and transferable adversarial examples from bayesian neural networks}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=hSQldaZNLbO},
}
Figure