Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AI

Casey Wall, Longwei Wang, Rodrigue Rizk, Kc SantosharXiv 2026 · arXiv

TL;DR

This paper exposes a critical reproducibility crisis in explainable AI: the widespread, "plug-and-play" application of Grad-CAM to Vision Transformers (ViTs) despite a fundamental architectural mismatch. By auditing 175 papers, the authors reveal that most studies fail to specify how they adapted this CNN-designed method to token-based representations. To fix this, they introduce a rigorous taxonomy that maps out the hidden methodological choices that drastically alter visual explanations.

What Problem Does This Paper Address?

Gradient-weighted Class Activation Mapping (Grad-CAM) is the go-to tool for generating heatmaps that show "where a model is looking." However, Grad-CAM was mathematically formulated for Convolutional Neural Networks (CNNs), which rely on spatial feature maps and clear channel dimensions.

Vision Transformers (ViTs) operate completely differently: they flatten images into sequences of patch tokens, mix information globally using self-attention, and pass representations through residual streams. Because ViTs lack native 2D convolutional feature maps, there is no single, obvious place to apply Grad-CAM. Researchers must make non-trivial engineering choices to force Grad-CAM onto a ViT. This paper addresses the fact that these choices are almost always left implicit, creating an illusion of standardized evaluation while masking massive variations in how heatmaps are actually generated.

Methods

The authors tackle this ambiguity through a two-pronged approach:

  1. A Systematic Literature Audit: They screened over 550 papers published from 2021 to 2026, ultimately auditing 175 papers that explicitly apply Grad-CAM or its close variants to token-based vision models. They coded these papers based on their task types, citation practices, and methodological justifications.
  2. A Descriptive Taxonomy: To formalize what it actually means to use Grad-CAM on a transformer, the authors mapped out the exact mathematical decision points required during implementation.

As shown in the architectural diagrams below, features can be extracted from multiple distinct locations within a standard self-attention or cross-attention block:

Standard ViT self-attention encoder block feature extraction locations
Multi-head cross-attention block feature extraction locations in vision-language models

The taxonomy categorizes these choices into:

  • Feature Extraction Locations: Token-feature methods (e.g., LayerNorm outputs $F_1$, MLP activations $F_5$) vs. Attention-map methods (e.g., softmax attention matrices $F_2$).
  • Token Handling: How the non-spatial [CLS] token is treated or discarded before reshaping tokens back into a 2D grid.
  • Aggregation Strategies: Whether attention heads are averaged before or after gradient weighting.

Datasets

Because this paper is a systematic audit and meta-analysis, it does not introduce a new model training dataset. Instead, it analyzes a corpus of 175 peer-reviewed papers collected from major computer vision and machine learning venues (including CVPR, ICCV, NeurIPS, WACV, and IEEE Access). For their qualitative demonstrations, the authors utilized a standard ViT-B/16 model evaluated on ImageNet-1k and a BLIP vision-language model evaluated on COCO.

Key Insights and Ideas

  • The 100+ Heatmap Paradox: For a single image and a standard ViT-B/16 model, there are well over 100 plausible ways to configure Grad-CAM based on layer depth, tensor location, token handling, and head aggregation. "Grad-CAM on a ViT" is not a single method; it is a massive space of alternatives.
  • Unchecked Researcher Degrees of Freedom: Because different implementation choices yield drastically different visual maps, researchers have the freedom to unconsciously or consciously cherry-pick the heatmap that looks the most "visually persuasive" or aligns best with their hypothesis, undermining the heatmap's value as objective scientific evidence.
  • The Vanishing Gradient Problem: In standard ViT classification, only the [CLS] token is fed into the final MLP head. Consequently, computing the gradient of a class score with respect to intermediate patch tokens often yields zero gradients at the final layer, forcing unacknowledged heuristic workarounds.

Key Results

  • 58% of Papers are Underspecified: 101 out of the 175 audited papers (58%) simply cited the original 2017 CNN-based Grad-CAM paper, providing zero mathematical or implementation details on how they adapted it to a transformer.
  • Only 15% Provide Justification: Only 26 papers either explicitly justified their transformer adaptation or cited a prior peer-reviewed work (such as ALBEF or Chefer et al.) that defined a ViT-specific implementation.
  • Extreme Visual Divergence: The authors experimentally demonstrate how dangerous this ambiguity is. By holding the model and input image constant but varying the taxonomy choices, they generated completely different heatmaps. Some configurations tightly localize the object, while others highlight random background noise or appear completely inverted.

This dramatic variation is clearly visible when comparing activations across different layers and token-weighting strategies:

Grad-CAM visualizations across token-feature locations F1, F4, F5, and F6
Grad-CAM visualizations across attention-based locations F2 and F3

Limitations and Future Work

The authors note that their literature corpus is a bounded snapshot, restricted to specific open-access repositories and keywords up to mid-2026. Crucially, this study evaluates methodological specification (how well authors report what they did) rather than empirical faithfulness (which adaptation is fundamentally the most accurate reflection of the model's inner workings).

The authors suggest that future work should focus on establishing strict reporting standards. They conclude with actionable checklists for both authors and reviewers, demanding that any future paper using Grad-CAM on transformers explicitly state the exact layer, tensor location, gradient target, and token-reshaping mechanics used.

Summary updated

Abstract

Gradient-weighted Class Activation Mapping (Grad-CAM) is widely used to visualize model decisions, but it was originally formulated for convolutional neural networks, where spatial feature maps and channel dimensions have clear architectural meanings. Vision Transformers (ViTs) do not provide the same structure, instead representing images through tokens, attention, residual streams, and multimodal interactions. This paper presents a systematic taxonomy and literature audit of how Grad-CAM and related methods are adapted, justified, and reported for ViT-based architectures. From an initial search of more than 550 papers, we identify 175 papers that apply Grad-CAM or Grad-CAM-adjacent methods to ViTs. We find that most papers do not provide a full mathematical or implementation-level account of how Grad-CAM is adapted to transformer representations. To characterize this gap, we introduce a descriptive taxonomy of ViT Grad-CAM adaptations that makes explicit the feature locations, gradient targets, spatial reconstruction steps, and aggregation choices that are often left implicit. This taxonomy is not intended to prescribe a single correct adaptation, but to clarify the range of methodological choices being made. The study shows that Grad-CAM on ViTs is often treated as a trivial extension of CNN-based Grad-CAM, despite requiring nontrivial choices that affect rigor, reproducibility, and interpretation.
BibTeX
@misc{wall2026grad,
  author = {Casey Wall and Longwei Wang and Rodrigue Rizk and KC Santosh},
  title = {{Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AI}},
  year = {2026},
  eprint = {2608.05258},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.05258},
}
Figure