Graph Neural Networks (GNNs) have become the standard tool for analyzing data structured as graphs, from molecular topologies to brain connectivity networks. However, because GNNs repeatedly mix information across neighboring nodes through message-passing layers, tracking exactly why a model made a specific prediction is notoriously difficult.
To solve this, researchers have adapted post-hoc interpretability methods from computer vision—specifically Class Activation Mapping (CAM) and gradient-based saliency maps—to graph structures. These techniques allow us to look inside a trained GNN and extract a localized "heat map" of the nodes, edges, or subgraphs that drove the model's decision.
From Spatial Grids to Graph Topologies
In a standard Convolutional Neural Network (CNN), Class Activation Mapping works because the final convolutional layers retain spatial information. By taking a weighted sum of these feature maps—where the weights are determined by the final classification layer—we can project the model's internal activations back onto the original pixel grid.
To apply this to graphs, we must bridge the gap between rigid pixel grids and irregular graph topologies. Byung-Hoon Kim et al. established a mathematical foundation for this translation in Understanding Graph Isomorphism Network for rs-fMRI Functional Connectivity Analysis. They demonstrated that certain GNN architectures act as mathematical duals to CNNs when the graph's adjacency matrix is treated as a shift operator. Because of this duality, the feature maps generated by the final graph convolutional layer correspond directly to individual nodes rather than spatial pixels.
There are two primary ways to extract these importance scores:
- Activation-Based (CAM): This approach leverages the weights of the final linear layer directly. By applying a global pooling layer across all nodes and multiplying the resulting feature vectors by the classification weights, the model projects importance scores back onto the input graph. In a comprehensive evaluation of GNN explainers, Benjamin Sanchez-Lengeling et al. showed in Evaluating Attribution for Graph Neural Networks that this global average pooling approach provides highly stable, reliable, and faithful explanations compared to many perturbation-based alternatives.
- Gradient-Based (Grad-CAM & Saliency): When a model uses more complex architectures where direct weight projection is impossible, gradients serve as a proxy for importance. By calculating the derivative of the target class score with respect to the node feature maps ($\frac{\partial y_c}{\partial h_i}$), we can quantify how sensitive the final prediction is to changes at a specific node. Houliang Zhou successfully adapted this technique in Interpretable Graph Convolutional Network Of Multi-Modality Brain Imaging For Alzheimer's Disease Diagnosis to isolate specific structural biomarkers from multi-modal brain scans.
[Input Graph] ---> [GNN Layers] ---> [Global Pooling] ---> [Class Score (yc)]
^ |
| v
[Saliency Map] <--- [Weight Activation / Compute Gradients] <-------+
(Node/Edge Weights) (dy_c / dh_i)
Expanding Beyond Nodes to Edges and Higher-Order Structures
While identifying important nodes is valuable, graph relationships are fundamentally defined by their connections. Therefore, the framework has been extended to capture edge-level importance and higher-order structural motifs.
Edge-Level Attribution
Instead of evaluating node features, edge attribution measures how information flow across specific links impacts the output. Tetsu Kasanishi et al. achieved this in Edge-Level Explanations for Graph Neural Networks by Extending Explainability Methods for Convolutional Neural Networks by computing gradients with respect to the message-passing operations themselves. If a small change to an edge's weight causes a massive shift in the final prediction, that edge is flagged as a critical communication pathway.
Higher-Order Structures
Real-world graphs often contain complex functional units, such as carbon rings in chemistry or cliques in social networks, that standard pairwise edges fail to capture fully. To address this, Akshit Sinha et al. proposed treating graphs as "cell complexes" in Higher Order Structures for Graph Explanations. Their framework, FORGE, assigns importance scores directly to these multi-node structures and then uses a hierarchical aggregation process to distribute the relevance scores back down to the constituent edges and nodes, preserving the context of the larger pattern.
Real-World Applications and Validation
Graph class activation mapping has transitioned from a theoretical diagnostic tool into a vital mechanism for scientific discovery across several domains.
Computational Pathology and Medicine
In digital pathology, models map cellular structures to identify cancer subtypes. Guillaume Jaume et al. utilized post-hoc graph attribution in Quantifying Explainers of Graph Neural Networks in Computational Pathology to verify that the subgraphs highlighted by GNN explainers align with actual biological traits (like cell size and shape) used by human pathologists, moving beyond ungrounded pixel heatmaps.
Bioinformatics and Drug Discovery
When predicting protein functions or drug interactions, understanding the "why" is a prerequisite for laboratory validation. Vladimir Gligorijević et al. applied gradient-weighted CAM to 3D structural protein graphs in Structure-based protein function prediction using graph convolutional networks, allowing their model to automatically pinpoint active and binding sites at the residue level without explicit site-level training. Similarly, Guadalupe Gonzalez et al. leveraged gradient-based attribution in Predicting anticancer hyperfoods with graph convolutional networks to trace global network predictions back to specific disease-fighting genes and biological pathways.
Neuroimaging
In neuroscience, identifying functional connectivity biomarkers is critical for tracking neurodegenerative diseases. Ying Zhang et al. combined spatiotemporal GNNs with class activation mapping in A novel spatiotemporal graph convolutional network framework for functional connectivity biomarkers identification of Alzheimer’s disease. By calculating the gradients of the output class relative to the final convolutional layer, they successfully mapped out the exact brain network defects that predict a patient's progression toward Alzheimer's disease.
From Visualization to Active Constraint
While graph CAM is primarily used as a post-hoc explanation tool, its underlying mechanism can also be integrated directly into the training loop to alter model behavior.
In Richly Activated Graph Convolutional Network for Action Recognition with Incomplete Skeletons, Yi-Fan Song et al. used CAM dynamically during training. Instead of just visualizing what the network looked at, they used the activation maps to identify the most influential spatial nodes (e.g., primary joint movements in a skeleton) and intentionally masked them out. This forced the network to find alternative, redundant discriminative features across the rest of the graph, ultimately making the model significantly more robust to missing or occluded data.
Key Foundational Work
Graph Saliency Maps Through Spectral Convolutional Networks: Application to Sex Classification with Brain Connectivity (2018) — contributed 2 concept elements to this direction.
Explainability Methods for Graph Convolutional Neural Networks (2019) — contributed 2 concept elements to this direction.
Richly Activated Graph Convolutional Network for Action Recognition with Incomplete Skeletons (2019) — contributed 1 concept elements to this direction.