ExAD: An Ensemble Approach for Explanation-based Adversarial Detection

Raj Vardhan, Ninghao Liu, Phakpoom Chinprutthiwong, Weijie Fu, Zhenyu Hu, Xia Ben Hu, Guofei GuarXiv 2021 · OpenReview

TL;DR

This paper introduces ExAD, a novel defense framework that uses an ensemble of "explanation maps" from different techniques to detect adversarial examples. By analyzing the consistency of these explanations, ExAD effectively identifies malicious inputs, even against sophisticated "whitebox" attacks that know the defense mechanism.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are increasingly used in security-sensitive applications, but they are vulnerable to adversarial examples. These are subtly modified inputs that cause DNNs to misclassify them, even though they look normal to humans. While many defenses exist, they often fail when attackers have full knowledge of the defense (whitebox attacks). This paper tackles the challenge of creating a robust adversarial detection system that can withstand these more advanced attacks.

Methods

ExAD's core idea is to leverage "explanation maps," which highlight which parts of an input (e.g., pixels in an image) are most important for a DNN's decision. The authors hypothesize that adversarial examples, even if they fool the DNN into a specific classification, will produce "abnormal" explanation maps compared to those generated for genuine examples of that class.

Here's how ExAD works:

  1. Ensemble of Explanations: ExAD uses a diverse set of explanation techniques (e.g., gradient-based, propagation-based) to generate multiple explanation maps for each input. This diversity is crucial to prevent an attacker from fooling all explanation methods simultaneously.
  2. Detector Networks: For each explanation technique and for each class, a dedicated "detector network" is trained. These detectors learn to distinguish between "normal" explanation maps (from legitimate inputs) and "abnormal" ones (from adversarial inputs).
  3. Test-Time Detection: When a new input arrives, ExAD generates its explanation maps using the ensemble. If any of the detector networks flags its corresponding explanation map as abnormal for the predicted class, the input is classified as adversarial.

The paper explores two types of detector models:

  • ExAD-CNN: Uses a CNN-based binary classifier trained on both normal and abnormal explanations.
  • ExAD-AE: Uses autoencoders to detect anomalies in explanation maps, requiring only normal explanations for training, making it potentially more robust to unseen attacks.

Datasets

The proposed ExAD framework was evaluated on three standard image datasets:

  • MNIST: A dataset of handwritten digits.
  • Fashion-MNIST (FMNIST): A dataset of clothing and accessory images, considered more challenging than MNIST due to its grayscale nature and more complex classes.
  • CIFAR-10: A dataset of small, color images representing 10 different object classes.

Key Insights and Ideas

  • Explanations as a Defense Vector: The paper demonstrates that the "explainability" of a model's decision can be a powerful tool for detecting adversarial attacks, not just for understanding model behavior.
  • Diversity is Key for Robustness: Using an ensemble of diverse explanation techniques is critical. An attacker can manipulate an input to fool one explanation method, but it's significantly harder to fool multiple methods with different underlying principles simultaneously.
  • Whitebox Attacks are Challenging but Not Invincible: While whitebox attacks, which know the defense, are a significant threat, ExAD shows that an ensemble of explanations can still achieve a respectable detection rate (over 88% in their experiments).
  • Transferability of Attacks: The research found interesting patterns in how attacks targeting one explanation technique transfer to others. For example, attacks on gradient-based methods were less effective against propagation-based detectors, and vice-versa, reinforcing the need for diversity.
  • Normal Explanations are Consistent: A key hypothesis, supported by experiments, is that genuine examples of a class produce relatively consistent explanation maps, which can be learned by detector models.

Key Results

  • Blackbox Attacks: ExAD achieved very high detection rates (often 100%) against six state-of-the-art blackbox adversarial attacks across all three datasets, with low false-positive rates (under 1.1%).
  • Whitebox Attacks: ExAD significantly limited the success rate of whitebox attacks, achieving a mean detection rate of over 88% across the three datasets.
  • Performance on Normal Data: The defense had a minimal impact on the accuracy of the underlying DNNs on normal data, with only a slight drop in accuracy. For example, on MNIST, the accuracy dropped from 99.15% to 98.26% (ExAD-CNN) or 98.54% (ExAD-AE).
  • Comparison with State-of-the-Art: ExAD performed competitively or better than existing detection methods like MagNet, Feature Squeezing, and LID, particularly in terms of robustness against various attacks and maintaining low false-positive rates.

Limitations and Future Work

  • Targeted Attacks Only: The current evaluation primarily focuses on targeted adversarial attacks. The authors suggest extending the evaluation to untargeted attacks.
  • Simultaneous Whitebox Attack Difficulty: The paper notes that simultaneously attacking multiple explanation techniques with a single optimization process is considerably difficult for adaptive adversaries. Future work could explore improving the efficiency and effectiveness of such attacks.
  • Fragility of Explanations: The paper discusses how explanations themselves can be fragile and manipulated. While ExAD aims to mitigate this through ensembles, applications that rely on explanations for critical decisions might need further research into explanation robustness.
  • Joint Detection: The authors suggest that combining ExAD-CNN and ExAD-AE (joint detection) could further boost generalizability and detection rates, especially against unknown attacks.

Summary updated

Abstract

Recent research has shown Deep Neural Networks (DNNs) to be vulnerable to adversarial examples that induce desired misclassifications in the models. Such risks impede the application of machine learning in security-sensitive domains. Several defense methods have been proposed against adversarial attacks to detect adversarial examples at test time or to make machine learning models more robust. However, while existing methods are quite effective under blackbox threat model, where the attacker is not aware of the defense, they are relatively ineffective under whitebox threat model, where the attacker has full knowledge of the defense. In this paper, we propose ExAD, a framework to detect adversarial examples using an ensemble of explanation techniques. Each explanation technique in ExAD produces an explanation map identifying the relevance of input variables for the model's classification. For every class in a dataset, the system includes a detector network, corresponding to each explanation technique, which is trained to distinguish between normal and abnormal explanation maps. At test time, if the explanation map of an input is detected as abnormal by any detector model of the classified class, then we consider the input to be an adversarial example. We evaluate our approach using six state-of-the-art adversarial attacks on three image datasets. Our extensive evaluation shows that our mechanism can effectively detect these attacks under blackbox threat model with limited false-positives. Furthermore, we find that our approach achieves promising results in limiting the success rate of whitebox attacks.
BibTeX
@misc{vardhan2021exad,
  author = {Raj Vardhan and Ninghao Liu and Phakpoom Chinprutthiwong and Weijie Fu and Zhenyu Hu and Xia Ben Hu and Guofei Gu},
  title = {{ExAD: An Ensemble Approach for Explanation-based Adversarial Detection}},
  year = {2021},
  eprint = {2103.11526},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.11526},
}
Figure