Trace and Detect Adversarial Attacks on CNNs using Feature Response Maps

Mohammadreza Amirian, Friedhelm Schwenker, Thilo Stadelmann

Abstract

The existence of adversarial attacks on convolutional neural networks (CNN) questions the fitness of such models for serious applications. These attacks manipulate an input image to evoke misclassification while remaining visually normal to humans, making them difficult to detect. This work proposes a novel detection method that tracks adversarial perturbations in feature responses—backpropagated activations of hidden layers. By using average local spatial entropy to analyze these maps, the method allows for automatic detection without altering network architecture. Experiments on ImageNet demonstrate high effectiveness against state-of-the-art attacks while remaining fully human-interpretable.

Introduction: The Adversarial Challenge

Deep learning models, particularly Convolutional Neural Networks (CNNs), are notoriously vulnerable to adversarial attacks. These involve small, often imperceptible tweaks to an input image that cause the model to confidently misclassify it. For instance, a model might see a "lampshade" instead of a "Norwich terrier" due to subtle noise.

Examples of adversarial attacks where subtle perturbations lead to drastic misclassifications.

While many defenses exist, they often suffer from performance degradation on clean images, require complex architectural changes, or lack transparency. This paper introduces a "plug-and-play" detection method that maintains original accuracy and provides a visual "audit trail" for human experts.

Methodology: Feature Response Maps and Spatial Entropy

The core of the proposed approach lies in Feature Response Maps. Using Guided Backpropagation, the researchers visualize which parts of an input image contribute most to the network's final decision.

A fundamental difference was observed in these maps:

  • Clean Images: The network's focus is "sharp" and concentrated on the actual object.
  • Adversarial Images: The network's focus becomes "diffuse" and scattered across the image as it attempts to process conflicting adversarial noise.

To automate detection, the researchers employed Average Local Spatial Entropy ($\bar{S}$). The feature response map is divided into $3 \times 3$ patches, and the entropy (a measure of randomness or complexity) is calculated for each. If the average entropy score exceeds a specific threshold, the input is flagged as adversarial.

Experimental Setup

The study utilized the ImageNet validation set (50,000 images, 1,000 classes) to test the detector against five attack types:

  • FGSM (Fast Gradient Sign Method): A fast, gradient-based attack.
  • Gradient Attack: A simple optimization-based attack.
  • One Pixel Attack: Modifying only a single pixel.
  • DeepFool: An iterative attack finding the closest decision boundary.
  • Boundary Attack: A reliable black-box attack.

Key Results

The method, tested on a VGG19 model, yielded several significant findings:

  • High Detection Accuracy: The detector achieved an AUC of 0.990 across several attack types.
  • Low False Positives: It successfully detected over 91% of adversarial samples with only a 1% False Positive Rate.
  • Sensitivity: The detector is particularly effective against "stronger" attacks like FGSM and Boundary attacks.
  • Zero Performance Impact: Because the detector is an external layer, the original classification accuracy for clean images remains 100% unaffected.

Discussion and Interpretability

A key advantage of this method is its interpretability. Because detection is based on visualization, a human "debugger" can inspect a flagged image and see that the network is focusing on background noise rather than the primary subject. This confirms the presence of an attack and provides insight into the model's failure mode.

Limitations and Future Work

The authors noted that detection is more challenging in binary classification tasks (e.g., Cat vs. Dog) compared to 1,000-class problems like ImageNet. In simpler tasks, the "gap" between classes is larger, sometimes requiring more visible perturbations to fool the model. Additionally, generating guided backpropagation maps adds computational overhead, which may impact real-time applications. Future work involves integrating these interpretability insights directly into the training process to develop inherently more robust models.

Summary updated

Abstract

The existence of adversarial attacks on convolutional neural networks (CNN) questions the fitness of such models for serious applications. The attacks manipulate an input image such that misclassification is evoked while still looking normal to a human observer -- they are thus not easily detectable. In a different context, backpropagated activations of CNN hidden layers -- "feature responses" to a given input -- have been helpful to visualize for a human "debugger" what the CNN "looks at" while computing its output. In this work, we propose a novel detection method for adversarial examples to prevent attacks. We do so by tracking adversarial perturbations in feature responses, allowing for automatic detection using average local spatial entropy. The method does not alter the original network architecture and is fully human-interpretable. Experiments confirm the validity of our approach for state-of-the-art attacks on large-scale models trained on ImageNet.
BibTeX
@inproceedings{amirian2018trace,
  author = {Mohammadreza Amirian and Friedhelm Schwenker and Thilo Stadelmann},
  title = {{Trace and Detect Adversarial Attacks on CNNs using Feature Response Maps}},
  booktitle = {ANNPR},
  year = {2018},
  doi = {10.1007/978-3-319-99978-4\_27},
}
Figure