Explainability-Driven Defense: Grad-CAM-Guided Model Refinement Against Adversarial Threats

Longwei Wang, Ifrat Ikhtear Uddin, Xiao Qin, Yang Zhou, Kc SantoshAAAI 2025 · DOI

In the arms race between deep learning models and adversarial attackers, the "black box" nature of neural networks is often their greatest weakness. Attackers exploit subtle, high-frequency patterns that models rely on but humans can’t even see. A new paper from researchers at the University of South Dakota and Auburn University suggests a clever way to turn the tables: using Explainable AI (XAI) not just to understand models, but to actively defend them.

TL;DR

This paper proposes a lightweight defense framework that uses Grad-CAM heatmaps to identify and mask "unstable" image features. By retraining models on these refined datasets, the authors significantly improve resilience against adversarial attacks like FGSM and PGD without the massive computational overhead of traditional adversarial training.

What Problem Does This Paper Address?

Convolutional Neural Networks (CNNs) are notoriously fragile. Small, intentional perturbations to an image—invisible to the human eye—can trick a state-of-the-art model into misclassifying a stop sign as a speed limit sign.

Existing defenses usually fall into two camps: Adversarial Training, which is effective but computationally expensive and prone to overfitting, and Input Transformations (like blurring or resizing), which often degrade the model's accuracy on clean, un-attacked images. The authors identify a gap: can we use the model's own "attention" to filter out the features that make it vulnerable?

Methods

The core idea is to use Grad-CAM (Gradient-weighted Class Activation Mapping) to see which parts of an image a model considers important. Once these regions are identified, the authors apply one of three masking strategies to "refine" the training data:

  1. Binary Feature Masking: A "hard" approach that keeps only the high-activation regions (the "hot spots" in a heatmap) and sets everything else to zero. This forces the model to ignore background noise.
  2. Gaussian-Blurred Masking: A "soft" approach that keeps the important regions sharp but applies a Gaussian blur to the rest. This preserves some spatial context while suppressing the high-frequency "chatter" that adversarial attacks often exploit.
  3. Difference-Based Masking: A comparative approach. The authors look at the features a baseline model uses versus a stronger model (ResNet-50). They then mask out features that are unique to the weaker baseline, assuming those are the "unstable" ones.

The workflow follows an iterative loop: train a baseline, extract feature importance, mask the dataset, and then retrain the model on this refined data.

Datasets

The researchers evaluated their framework on three widely recognized benchmarks:

  • CIFAR-10: 60,000 images across 10 classes.
  • CIFAR-100: A more complex version with 100 classes.
  • CIFAR-10C: A "corrupted" version of CIFAR-10 designed to test robustness against real-world noise like fog, snow, and blur.

Key Insights and Ideas

  • XAI as a Proactive Tool: Traditionally, Grad-CAM is a post-hoc debugging tool used to ask, "Why did the model do that?" This paper pivots XAI into a proactive defense mechanism.
  • The Context Trade-off: While Binary Masking is great for simple attacks, it can destroy the "global" structure of an image. The authors found that Gaussian-blurred masking is superior for defending against iterative, aggressive attacks (like PGD) because it maintains the overall coherence of the scene.
  • Eliminating "Unstable" Features: The success of the Difference-Based Masking suggests that models often fail because they latch onto features that more robust architectures (like ResNet-50) have already learned to ignore.

Key Results

The defense was tested against two major attack types: the Fast Gradient Sign Method (FGSM) and the more potent Projected Gradient Descent (PGD).

  • FGSM Resilience: Both Binary and Difference-based masking provided consistent accuracy gains across various perturbation levels ($\epsilon$).
  • PGD Resilience: Gaussian-blurred masking was the clear winner here. In a striking result on the CIFAR-10 dataset with a high perturbation level ($\epsilon = 0.1$), the Gaussian-blurred model maintained 9.89% accuracy, while the baseline model dropped to 0.0%.
  • Efficiency: Unlike adversarial training, which requires generating new attack examples for every training step, this masking approach is a one-time preprocessing and retraining step, making it much more scalable.

Limitations and Future Work

The authors acknowledge a few hurdles:

  • Aggressive Removal: Binary masking can sometimes be too aggressive, removing features that might be necessary for generalizing to new, unseen datasets.
  • Scaling Up: While effective on CIFAR, the computational cost of generating Grad-CAM masks for massive datasets like ImageNet remains a challenge.
  • Future Directions: The team suggests exploring "counterfactual masking" (masking what the model shouldn't see) and investigating how varying the intensity of the Gaussian blur could further optimize the balance between clean accuracy and adversarial robustness.

Summary updated

Abstract

Deep learning models have excelled in tasks like image recognition and autonomous systems but remain vulnerable to adversarial attacks and spurious correlations, limiting their reliability in real-world and safety-critical settings. To address these challenges, we propose a novel framework that leverages explainable Artificial Intelligence (XAI) to enhance the robustness of Convolutional Neural Networks. Our approach integrates Grad-CAM insights into the model refinement process, guiding feature masking to reduce reliance on irrelevant or misleading features. We introduce three masking strategies: (1) binary masking to retain high-activation regions, (2) Gaussian-blurred masking to preserve contextual information while reducing noise, and (3) difference-based masking to remove unstable features unique to the baseline model. We evaluate these strategies against two common adversarial attack methods—Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD). Results show that all three strategies improve FGSM accuracy, with binary and difference-based masking providing consistent gains across perturbation levels. Gaussian-blurred masking delivers the highest improvement in PGD accuracy, particularly at higher perturbation strengths.
BibTeX
@inproceedings{wang2025explainability,
  author = {Longwei Wang and Ifrat Ikhtear Uddin and Xiao Qin and Yang Zhou and KC Santosh},
  title = {{Explainability-Driven Defense: Grad-CAM-Guided Model Refinement Against Adversarial Threats}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaaiss.v6i1.36024},
}
Figure