Where Does the Robustness Come from?: A Study of the Transformation-based Ensemble Defence

Chang Liao, Yao Cheng, Chengfang Fang, Jie Shi

TL;DR

This paper investigates the robustness of transformation-based ensemble defenses against adversarial attacks. It reveals that the robustness gained is primarily from irreversible transformations, not the ensemble itself, and that simply increasing the number of sub-models doesn't guarantee better defense.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are highly susceptible to adversarial examples (AEs) – subtly modified inputs that cause misclassifications. While ensemble methods have been proposed to improve DNN robustness, a specific type called "transformation-based ensemble defense" has shown promise. This defense trains multiple sub-models on data pre-processed with various transformations. However, it's unclear whether the observed robustness stems from the transformations themselves or the ensemble aggregation. This paper aims to dissect this by analyzing the contributions of reversible vs. irreversible transformations and the ensemble effect.

Methods

The authors introduce two novel adaptive attacks to rigorously evaluate transformation-based ensembles:

  • Transferability-based Adaptive Attack (TAA): This attack targets the "weakest" sub-model in the ensemble, identified by the transferability of AEs generated on it to other sub-models. The idea is that attacking the most vulnerable link can compromise the entire ensemble.
  • Perturbation Aggregation Attack (PAA): This attack aggregates adversarial perturbations generated on multiple sub-models using various strategies (MaxP, AvgP, MVoteP, WSumP). This allows for a more comprehensive attack, especially when individual sub-models are resistant.

They also categorize transformations into reversible and irreversible, designing experiments to isolate their impact.

Datasets

The experiments were conducted on the MNIST dataset, a standard benchmark for image classification tasks, specifically handwritten digits. The authors split the original testing data into validation and testing sets.

Key Insights and Ideas

  • Robustness Source: The primary source of robustness in transformation-based ensembles is the irreversible transformations applied to the data, not the aggregation of multiple models.
  • Limited Ensemble Benefit: Simply increasing the number of sub-models in a transformation-based ensemble does not significantly improve robustness.
  • Transferability is Key (and a Weakness): Adversarial examples can still transfer between models trained on data pre-processed with different reversible transformations. This transferability is a significant vulnerability for ensembles relying solely on reversible transformations.
  • Attack Sophistication Matters: Gradient-based attacks (like FGSM and PGD) tend to have higher transferability than optimization-based attacks (like C&W and DeepFool), making them more potent against certain ensemble configurations.

Key Results

  • Reversible Ensembles Vulnerable: Ensembles composed solely of sub-models trained on data with reversible transformations are vulnerable to TAA and PAA. For instance, PGD-based TAA could reduce ensemble accuracy to around 15% by targeting the "shift top left" sub-model.
  • Irreversible Transformations Boost Robustness: Introducing irreversible transformations into an ensemble, even when combined with reversible ones, provides some robustness. However, this robustness is limited and doesn't match the "benign" (no attack) performance.
  • Limited Gain from Irreversible Ensembles: Ensembles composed only of irreversible transformation sub-models showed slightly better accuracy under attack than hybrid ensembles, but still significantly lower than benign performance. Increasing the number of irreversible sub-models did not yield substantial improvements.
  • PAA Effectiveness: PAA, especially with WSumP or AvgP aggregation, could significantly drop ensemble accuracy to around 20% when using PGD as the base attack.

Limitations and Future Work

  • Limited Robustness: The paper concludes that transformation-based ensemble defense, as studied, cannot provide sufficient robustness for critical applications.
  • Need for Continuous Effort: The authors emphasize the ongoing "arms race" between attack and defense, highlighting the necessity for continuous research in defense mechanisms.
  • Fine-tuning WSumP: The authors suggest that further fine-tuning of the weights in the WSumP aggregation strategy could potentially improve attack effectiveness.
  • Exploring Other Transformations: While the paper categorizes transformations, a deeper dive into the specific properties of irreversible transformations that contribute to robustness could be a future direction.

Summary updated

Abstract

This paper aims to provide a thorough study on the effectiveness of the transformation-based ensemble defence for image classification and its reasons. It has been empirically shown that they can enhance the robustness against evasion attacks, while there is little analysis on the reasons. In particular, it is not clear whether the robustness improvement is a result of transformation or ensemble. In this paper, we design two adaptive attacks to better evaluate the transformation-based ensemble defence. We conduct experiments to show that 1) the transferability of adversarial examples exists among the models trained on data records after different reversible transformations; 2) the robustness gained through transformation-based ensemble is limited; 3) this limited robustness is mainly from the irreversible transformations rather than the ensemble of a number of models; and 4) blindly increasing the number of sub-models in a transformation-based ensemble does not bring extra robustness gain.
BibTeX
@inproceedings{liao2020where,
  author = {Chang Liao and Yao Cheng and Chengfang Fang and Jie Shi},
  title = {{Where Does the Robustness Come from?: A Study of the Transformation-based Ensemble Defence}},
  booktitle = {Aisec@ccs},
  year = {2020},
  doi = {10.1145/3411508.3421380},
}
Figure