Abstract
The existence of adversarial attacks on convolutional neural networks (CNN) questions the fitness of such models for serious applications. These attacks manipulate an input image to evoke misclassification while remaining visually normal to humans, making them difficult to detect. This work proposes a novel detection method that tracks adversarial perturbations in feature responses—backpropagated activations of hidden layers. By using average local spatial entropy to analyze these maps, the method allows for automatic detection without altering network architecture. Experiments on ImageNet demonstrate high effectiveness against state-of-the-art attacks while remaining fully human-interpretable.
Introduction: The Adversarial Challenge
Deep learning models, particularly Convolutional Neural Networks (CNNs), are notoriously vulnerable to adversarial attacks. These involve small, often imperceptible tweaks to an input image that cause the model to confidently misclassify it. For instance, a model might see a "lampshade" instead of a "Norwich terrier" due to subtle noise.
While many defenses exist, they often suffer from performance degradation on clean images, require complex architectural changes, or lack transparency. This paper introduces a "plug-and-play" detection method that maintains original accuracy and provides a visual "audit trail" for human experts.
Methodology: Feature Response Maps and Spatial Entropy
The core of the proposed approach lies in Feature Response Maps. Using Guided Backpropagation, the researchers visualize which parts of an input image contribute most to the network's final decision.
A fundamental difference was observed in these maps:
- Clean Images: The network's focus is "sharp" and concentrated on the actual object.
- Adversarial Images: The network's focus becomes "diffuse" and scattered across the image as it attempts to process conflicting adversarial noise.
To automate detection, the researchers employed Average Local Spatial Entropy ($\bar{S}$). The feature response map is divided into $3 \times 3$ patches, and the entropy (a measure of randomness or complexity) is calculated for each. If the average entropy score exceeds a specific threshold, the input is flagged as adversarial.
Experimental Setup
The study utilized the ImageNet validation set (50,000 images, 1,000 classes) to test the detector against five attack types:
- FGSM (Fast Gradient Sign Method): A fast, gradient-based attack.
- Gradient Attack: A simple optimization-based attack.
- One Pixel Attack: Modifying only a single pixel.
- DeepFool: An iterative attack finding the closest decision boundary.
- Boundary Attack: A reliable black-box attack.
Key Results
The method, tested on a VGG19 model, yielded several significant findings:
- High Detection Accuracy: The detector achieved an AUC of 0.990 across several attack types.
- Low False Positives: It successfully detected over 91% of adversarial samples with only a 1% False Positive Rate.
- Sensitivity: The detector is particularly effective against "stronger" attacks like FGSM and Boundary attacks.
- Zero Performance Impact: Because the detector is an external layer, the original classification accuracy for clean images remains 100% unaffected.
Discussion and Interpretability
A key advantage of this method is its interpretability. Because detection is based on visualization, a human "debugger" can inspect a flagged image and see that the network is focusing on background noise rather than the primary subject. This confirms the presence of an attack and provides insight into the model's failure mode.
Limitations and Future Work
The authors noted that detection is more challenging in binary classification tasks (e.g., Cat vs. Dog) compared to 1,000-class problems like ImageNet. In simpler tasks, the "gap" between classes is larger, sometimes requiring more visible perturbations to fool the model. Additionally, generating guided backpropagation maps adds computational overhead, which may impact real-time applications. Future work involves integrating these interpretability insights directly into the training process to develop inherently more robust models.
Summary updated