On Saliency Maps and Adversarial Robustness

Puneet Mangla, Vedant Singh, Vineeth N. Balasubramanian

TL;DR

This paper introduces Saliency-based Adversarial Training (SAT), a novel method that leverages saliency maps to improve the adversarial robustness of deep neural networks. By using saliency maps to guide the generation of adversarial perturbations during training, SAT can enhance robustness without requiring additional computational cost for generating these perturbations, especially when using readily available dataset annotations like bounding boxes.

What Problem Does This Paper Address?

Deep neural networks (DNNs) are increasingly used in critical applications, making their robustness against adversarial attacks and their interpretability crucial. While previous research has explored the link between these two aspects, often showing that robust models have more interpretable saliency maps, this paper flips the script. It addresses the challenge of improving adversarial robustness by using saliency maps as a tool, rather than just observing their properties. The authors aim to bridge the gap between interpretability and robustness by proposing a practical method to enhance a model's resilience to adversarial perturbations, even when using "weak" saliency information like bounding boxes.

Methods

The core of the paper is the Saliency-based Adversarial Training (SAT) method. The motivation stems from the observation that adversarial perturbations at later stages of training often target class-discriminative regions of an image, which are also highlighted by saliency maps.

Here's how it works:

  • Saliency Map Generation: Saliency maps can be obtained from existing dataset annotations (like bounding boxes or segmentation masks) or from a pre-trained "teacher" model. The paper shows that even "weak" saliency maps are effective.
  • Perturbation Direction: The method leverages the insight that adversarial perturbations are often aligned with the gradient of the class score with respect to the input. Specifically, they use the negative gradient of the true class score (which is essentially the saliency map for the true class) as the direction for generating adversarial perturbations.
  • Stochastic Perturbations: To mimic the evolving nature of adversarial perturbations during training, SAT introduces a stochastic element. The perturbation direction is a combination of random noise and the saliency-derived direction, controlled by a hyperparameter a. This allows the method to adapt as the model learns.
  • Integration with Existing Methods: SAT can be used independently or integrated with established adversarial training techniques like Projected Gradient Descent (PGD) and TRADES, leading to further performance boosts.

Datasets

The authors evaluated their method on several widely used datasets:

  • CIFAR-10: A standard dataset for image classification with 10 classes.
  • CIFAR-100: A more challenging version of CIFAR-10 with 100 classes.
  • Tiny ImageNet: A larger dataset with 200 classes.
  • Flower-17: A dataset specifically for flower classification with 17 categories.

For Tiny ImageNet and Flower-17, the paper highlights the use of pre-existing bounding box and segmentation mask annotations as "weak" saliency maps, demonstrating that SAT can leverage these without additional annotation effort.

Key Insights and Ideas

  • Saliency Maps as a Tool for Robustness: The paper demonstrates that saliency maps aren't just for understanding models; they can actively improve a model's defense against adversarial attacks.
  • Weak Saliency is Sufficient: Surprisingly, even simple annotations like bounding boxes, often overlooked, can be effectively used as saliency maps to boost adversarial robustness. This significantly reduces the practical barrier to entry.
  • Finer Saliency, Better Robustness: The quality of the saliency map matters. Finer and more accurate saliency maps (e.g., from adversarially trained teacher models or better explanation methods) lead to more robust models.
  • Synergy with Existing Defenses: SAT isn't a replacement for existing adversarial training methods but a powerful complement. Integrating SAT with PGD and TRADES further enhances their performance.
  • Class-Discriminative Regions are Key: The core idea that adversarial perturbations target class-discriminative regions, which are also highlighted by saliency maps, is a fundamental insight driving the method.

Key Results

  • Improved Robustness: SAT consistently improved adversarial robustness across all tested datasets (CIFAR-10, CIFAR-100, Tiny ImageNet, Flower-17). For instance, on CIFAR-10, SAT achieved improvements of 2-4% in adversarial accuracy compared to baselines, even when using weak saliency maps.
  • Efficiency: When using pre-existing annotations as saliency maps, SAT incurs no additional computational cost for generating perturbations, making it more efficient than traditional adversarial training methods.
  • Synergistic Gains: Integrating SAT with PGD and TRADES (resulting in PGD-SAT and TRADES-SAT) led to further improvements, demonstrating the method's additive benefit. For example, PGD-SAT showed a 1.53% improvement over vanilla PGD.
  • Saliency Map Quality Matters: Experiments showed that using saliency maps from adversarially trained teacher models or from finer explanation methods (like Guided Backpropagation) yielded better robustness than those from normally trained teachers or weaker methods.
  • Time Efficiency: PGD-SAT and TRADES-SAT were found to be ~50-70% faster than vanilla PGD and TRADES, respectively, while achieving superior performance.

Limitations and Future Work

  • Saliency Map Quality: The paper acknowledges that the quality of saliency maps directly impacts the effectiveness of SAT. While they show weak saliency can work, stronger saliency maps generally lead to better results.
  • Hyperparameter Tuning: The performance of SAT is influenced by hyperparameters like a (controlling the mix of random noise and saliency-derived perturbations) and ε₀ (related to perturbation distribution). The paper presents some analysis but suggests further exploration.
  • Future Directions: The authors suggest several avenues for future work:
  • Reasoning about the class closest to the ground truth in terms of decision boundaries.
  • Improving the estimation of adversarial perturbations.
  • Exploring ways to integrate multiple saliency maps from different sources to potentially cover a wider range of adversarial perturbations.
  • The need for stronger saliency map annotations in vision datasets to further enhance robustness with minimal overhead.

Summary updated

Abstract

A very recent trend has emerged to couple the notion of interpretability and adversarial robustness, unlike earlier efforts that focus solely on good interpretations or robustness against adversaries. Works have shown that adversarially trained models exhibit more interpretable saliency maps than their non-robust counterparts, and that this behavior can be quantified by considering the alignment between the input image and saliency map. In this work, we provide a different perspective to this coupling and provide a method, Saliency based Adversarial training (SAT), to use saliency maps to improve the adversarial robustness of a model. In particular, we show that using annotations such as bounding boxes and segmentation masks, already provided with a dataset, as weak saliency maps, suffices to improve adversarial robustness with no additional effort to generate the perturbations themselves. Our empirical results on CIFAR-10, CIFAR-100, Tiny ImageNet, and Flower-17 datasets consistently corroborate our claim, by showing improved adversarial robustness using our method. We also show how using finer and stronger saliency maps leads to more robust models, and how integrating SAT with existing adversarial training methods, further boosts the performance of these existing methods.
BibTeX
@inproceedings{mangla2020saliency,
  author = {Puneet Mangla and Vedant Singh and Vineeth N. Balasubramanian},
  title = {{On Saliency Maps and Adversarial Robustness}},
  booktitle = {ECML/PKDD},
  year = {2020},
  url = {https://openreview.net/forum?id=_gsX-rkgTo},
}
Figure