Demystifying the Adversarial Robustness of Random Transformation Defenses

Chawin Sitawarin, Zachary J. Golan-Strieb, David A. Wagner

TL;DR

Random Transformation (RT) defenses, once thought to be highly robust against adversarial attacks, are largely a "paper tiger." By developing a stronger, tailored attack and using Bayesian Optimization to tune the defense, researchers demonstrated that adversarial accuracy on benchmarks like Imagenette drops from a perceived 70% to a measly 6%.

What Problem Does This Paper Address?

Deep neural networks are notoriously fragile. Tiny, intentional perturbations to an input—known as adversarial examples—can cause them to fail spectacularly, raising serious security concerns for applications like autonomous driving. To counter this, researchers proposed Random Transformation (RT) defenses.

These defenses, such as the well-known "Barrage of Random Transforms" (BaRT), apply a sequence of random image augmentations (e.g., cropping, blurring, or color jittering) before the image is classified.

Architecture of a Random Transformation defense

Because these defenses introduce randomness and often rely on non-differentiable operations (like JPEG compression), evaluating their true robustness is notoriously difficult. Previous studies suggested they were highly effective, but this paper argues that those results were an artifact of sub-optimal attacks. The authors set out to determine if RT defenses are truly robust or if the community simply hasn't been trying hard enough to break them.

Methods

The authors approached the problem from two sides: building the best possible defense and then breaking it with the strongest possible attack.

  1. Defense Optimization: To ensure they weren't attacking a "straw man," the authors used Bayesian Optimization (BO) to find the most robust set of transformations and hyperparameters. This computationally expensive step "tuned" the defense to its absolute peak performance.
  2. The New Attack: They introduced a state-of-the-art attack tailored specifically for stochastic defenses. Key components include:
  3. Linear Loss on Logits: Instead of standard Cross-Entropy loss, they used a linear loss function, which significantly reduces the variance of noisy gradient estimates.
  4. Skip Gradient Method (SGM): Borrowed from transfer-attack literature, this technique prioritizes gradients through skip connections in ResNets, helping the attack "transfer" across the different random versions of the image.
  5. Aggregated Momentum (AggMo): A more stable optimizer than standard PGD, designed to handle the high-variance gradients inherent in randomized models.
  6. Differentiable Approximations: They replaced non-differentiable transforms with smooth, differentiable versions. This allowed for exact gradient flow rather than relying on the often-unreliable Backward-Pass Differentiable Approximation (BPDA).

Datasets

The study utilized two primary datasets:

  • CIFAR-10: A standard small-image benchmark (32x32).
  • Imagenette: A 10-class subset of ImageNet (224x224). The authors chose this because many image transformations behave differently on high-resolution images, and full ImageNet was too computationally expensive for their extensive Bayesian Optimization runs.

Key Insights and Ideas

  • BPDA is a Trap: Many researchers use BPDA to "bypass" non-differentiable layers by using a neural network to approximate the layer's behavior. The authors found this is highly ineffective for RT defenses. These approximator networks struggle to capture spatial shifts like cropping or complex noise, leading to a false sense of security.
  • Variance is the Real Enemy: In a randomized defense, the gradient is a moving target. The "aha moment" of this paper is that the success of an attack depends less on the mean of the gradient and more on its variance.
Effectiveness of linear loss vs. cross-entropy on adversarial accuracy
  • RT as an Infinite Ensemble: An RT defense can be viewed as an ensemble of an infinite number of models (one for every possible random seed). Consequently, techniques designed to improve adversarial transferability (like SGM) are incredibly effective at breaking these defenses.

Key Results

The results serve as a wake-up call for the robustness community:

  • The Collapse of BaRT: On the Imagenette dataset, the baseline attack (Expectation over Transformation or EoT) only reduced accuracy to 70.79%. The authors' new attack crushed that accuracy down to 6.34%.
  • AdvRT Gains: The authors tried combining RT with adversarial training (AdvRT). While this provided a significant boost on CIFAR-10 (reaching 41.3% adversarial accuracy), it still lagged behind state-of-the-art deterministic defenses like Madry’s adversarial training.
  • Resolution Matters: RT defenses appeared more promising on CIFAR-10 than on Imagenette, suggesting that the "security" provided by random transforms doesn't scale well to higher-resolution, more complex images.

Limitations and Future Work

The primary limitation of this study is computational cost. Running Bayesian Optimization to find the "best" defense took days of GPU time, making it difficult to scale the evaluation to the full ImageNet dataset.

The authors conclude that for any future RT defense to be viable, it must be built using only differentiable transformations. This would allow for proper adversarial training, which remains the most reliable path to robustness. They suggest that future work should focus on finding specific combinations of differentiable transforms that can actually compete with, rather than just supplement, traditional adversarial training.

Summary updated

Abstract

Neural networks’ lack of robustness against attacks raises concerns in security-sensitive settings such as autonomous vehicles. While many countermeasures may look promising, only a few withstand r...
BibTeX
@inproceedings{sitawarin2022demystifying,
  author = {Chawin Sitawarin and Zachary J. Golan-Strieb and David A. Wagner},
  title = {{Demystifying the Adversarial Robustness of Random Transformation Defenses}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=sV_MrbJPKR},
}
Figure