Demystifying the Adversarial Robustness of Random Transformation Defenses

Chawin Sitawarin, Zachary Golan-Strieb, David Wagner

TL;DR

This paper reveals that popular "random transformation" defenses against adversarial attacks, like BaRT, are not as robust as previously believed due to flawed evaluation methods. The authors introduce a stronger attack and demonstrate that even optimized random transformation defenses are vulnerable, suggesting a need for differentiable transformations and more rigorous evaluation in this domain.

What Problem Does This Paper Address?

Deep neural networks are increasingly used in safety-critical applications like autonomous driving, but they suffer from a significant vulnerability: adversarial examples. These are subtly modified inputs that can cause models to make incorrect predictions. Random Transformation (RT) defenses, which apply random changes to inputs during inference, have shown promise in mitigating these attacks. However, the authors argue that these defenses haven't been rigorously evaluated. Their stochastic nature makes standard attack methods less effective, and existing evaluation techniques, like the BPDA attack, can overestimate robustness. This paper aims to demystify the true robustness of RT defenses by developing better evaluation methods and a stronger attack.

Methods

The core of this paper involves two main thrusts: improving the evaluation of RT defenses and developing a stronger attack against them.

  • Rethinking Evaluation: The authors first demonstrate that the BPDA (Backward-Pass Differentiable Approximation) attack, commonly used to evaluate RT defenses with non-differentiable transformations, is ineffective. They show it significantly overestimates robustness. They propose using only differentiable transformations for more reliable evaluations and for compatibility with adversarial training.
  • Constructing a Stronger Attack: To rigorously evaluate RT defenses, the paper introduces a new state-of-the-art attack. This attack leverages insights from transfer attacks and advanced stochastic optimization algorithms. Key components include:
  • Differentiable Approximations: For non-differentiable transformations (like JPEG compression or color space conversions), they use smooth, differentiable approximations not just for the backward pass (as in BPDA) but also for the forward pass.
  • Optimized Decision Rules and Loss Functions: They experiment with different decision rules (e.g., majority vote, average softmax, average logits) and loss functions (cross-entropy, linear loss) for generating adversarial examples, finding that linear loss on logits is most effective.
  • Variance Reduction Techniques: Recognizing that high gradient variance is a bottleneck for attacking stochastic models, they employ techniques like skip connections (SGM) and an optimizer called Aggregated Momentum (AggMo) to reduce variance and improve attack convergence.
  • Hyperparameter Optimization: To ensure they are evaluating the most robust RT defense, they use Bayesian Optimization (BO) to tune the defense's hyperparameters, such as the types of transformations and their strengths.

Datasets

The paper uses the following datasets:

  • Imagenette: A ten-class subset of ImageNet, chosen for its similarity to ImageNet (where BaRT was originally evaluated) and its manageable size for extensive experimentation.
  • CIFAR-10: A standard benchmark dataset for adversarial robustness research.

Key Insights and Ideas

  • BPDA is Misleading: The common practice of using BPDA to approximate gradients for non-differentiable transformations in RT defenses is shown to be fundamentally flawed, leading to an overly optimistic assessment of robustness.
  • Differentiability is Key: For reliable evaluation and effective adversarial training, RT defenses should ideally rely on differentiable transformations. This allows for more precise gradient-based attacks.
  • Gradient Variance is the Enemy: The stochastic nature of RT defenses leads to high variance in gradient estimates, which significantly hinders the effectiveness of gradient-based attacks. Techniques that reduce this variance are crucial for developing stronger attacks.
  • Optimized Attacks are Devastating: The authors' new attack, incorporating differentiable approximations, optimized loss functions, and variance reduction techniques, dramatically outperforms existing attacks, revealing that even carefully tuned RT defenses are not robust.
  • Adversarial Training Needs Stronger Attacks: Combining RT defenses with adversarial training shows limited gains, suggesting that current attacks are not strong enough to effectively train robust models in this context.

Key Results

  • BPDA Overestimation: The BPDA attack resulted in adversarial accuracies of 36.49% and 52.32% on BaRT models, while the "Combo" attack (using exact gradients where possible) achieved much lower accuracies of 25.24% and 65.28% respectively, indicating BPDA overestimates robustness by a significant margin.
  • New Attack Effectiveness: The authors' proposed attack reduced adversarial accuracy on a fine-tuned RT defense from 70.79% (baseline) to 29.91% on Imagenette, a substantial improvement (over 2.3x stronger). On CIFAR-10, it reduced accuracy from 33.83% to 6.34%, a nearly 5.4x improvement.
  • RT Defense Vulnerability: Even an RT defense with optimized hyperparameters (tuned via Bayesian Optimization) showed very little robustness against the new attack, achieving only 29.91% adversarial accuracy on Imagenette.
  • AdvRT Gains: Adversarially training an RT defense with their new attack (AdvRT) resulted in a robustness gain on CIFAR-10 (from 29.91% to 41.30%), but still lagged behind robust deterministic models. On Imagenette, AdvRT showed minimal improvement over the base RT defense.

Limitations and Future Work

  • Computational Cost: Rigorous evaluation and hyperparameter tuning (like Bayesian Optimization) for RT defenses are computationally very expensive, requiring significant resources and time.
  • Adversarial Training Effectiveness: The paper highlights that current attacks are not strong enough to effectively train robust RT-based models through adversarial training. Future work should focus on developing attacks that converge faster and are more effective for this purpose.
  • Scalability of Attacks: While the new attack is powerful, its computational demands might limit its direct application in large-scale adversarial training.
  • Non-Differentiable Transformations: While the paper recommends using only differentiable transformations, some inherently non-differentiable operations might still be desirable. Future research could explore better ways to handle these.
  • Broader RT Defense Landscape: The paper focuses on BaRT and general RT defenses. Further investigation into other RT defense architectures and their robustness against advanced attacks would be valuable.

Summary updated

Abstract

Current machine learning models suffer from evasion attacks (i.e., adversarial examples) raising concerns in security-sensitive settings such as autonomous vehicles. While many countermeasures may look promising, only a few withstand rigorous evaluation. Recently, defenses using random transformations (RT) have shown impressive results, particularly BaRT (Raff et al. 2019) on ImageNet. However, this type of defense has not been rigorously evaluated, leaving its robustness properties poorly understood. The stochasticity of these models also makes evaluation more challenging and many proposed attacks on deterministic models inapplicable. First, we show that the BPDA attack (Athalye, Carlini, and Wagner 2018) used in BaRT’s evaluation is ineffective and likely overestimates its robustness. We then attempt to construct the strongest possible RT defense through the informed selection of transformations and Bayesian optimization for tuning their parameters. Furthermore, we create the strongest possible attack to evaluate our RT defense. Our new attack vastly outperforms the baseline, reducing the accuracy by $83\%$ compared to the $19\%$ reduction by the commonly used EoT attack ($4.3\times$ improvement). Our result indicates that the RT defense on Imagenettedataset (a ten-class subset of ImageNet) is not robust against adversarial examples. Extending the study further, we use our new attack to adversarially train RT defense (called AdvRT). However, the attack is still not sufficiently strong, and thus, the AdvRT model is no more robust than its RT counterpart. In the process of formulating our defense and attack, we perform several ablation studies and uncover insights that we hope will broadly benefit scientific communities studying stochastic neural networks and their robustness properties
BibTeX
@inproceedings{sitawarin2022demystifying,
  author = {Chawin Sitawarin and Zachary Golan-Strieb and David Wagner},
  title = {{Demystifying the Adversarial Robustness of Random Transformation Defenses}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=sV_MrbJPKR},
}
Figure