Towards Interpretable Semantic Segmentation via Gradient-Weighted Class Activation Mapping (Student Abstract)

Kira Vinogradova, Alexandr Dibrov, Gene MyersAAAI 2020 · DOI

Introduction

Deep learning models, particularly Convolutional Neural Networks (CNNs), have achieved state-of-the-art performance in semantic segmentation tasks. However, these models often operate as "black boxes," providing little insight into the reasoning behind their pixel-level predictions. While interpretability tools like Grad-CAM exist for image classification, they lack the granularity required for dense prediction tasks. This paper introduces SEG-GRAD-CAM, an extension of the Grad-CAM algorithm designed specifically to explain semantic segmentation decisions at the pixel or regional level.

Methodology

SEG-GRAD-CAM adapts the Gradient-weighted Class Activation Mapping approach by redefining the target score used for gradient calculation. In standard Grad-CAM, the importance of a feature map is derived from the gradient of a single class logit. SEG-GRAD-CAM instead utilizes a sum of logits over a specific set of pixels, denoted as $M$.

This formulation allows for high flexibility in the scope of explanation:

  • Single Pixel: $M$ consists of a single coordinate, explaining the classification of that specific point.
  • Object Instance: $M$ includes all pixels belonging to a specific instance (e.g., a single car).
  • Global Class: $M$ encompasses every pixel of a certain category (e.g., all "Sky" pixels) to identify broader contextual cues.

The researchers implemented this method using a U-Net architecture, focusing on the bottleneck layer where semantic representations are most compressed.

Experimental Results and Discussion

The method was evaluated on the Cityscapes dataset, which features high-resolution urban street scenes. Qualitative analysis demonstrates that SEG-GRAD-CAM effectively localizes the features driving segmentation decisions.

Comprehensive visualization of SEG-GRAD-CAM for a pixel in the 'Flat' class, including input, heatmap, ground truth, and prediction.

Key Insights

  • Contextual Dependency: Heatmaps for individual pixels often highlight regions far beyond their theoretical receptive field. This suggests the network leverages global environmental context to make local classifications.
  • Learned Biases: For the "Sky" class, the model frequently attends to "Nature" (tree canopies), indicating it has learned to associate the jagged boundaries of trees with the presence of sky above.
  • Layer Sensitivity: Bottleneck layers proved more informative for visualization than layers closer to the output, as they capture higher-level semantic concepts rather than simple edges.

Conclusion and Future Work

SEG-GRAD-CAM provides a viable framework for interpreting dense prediction models. While current results are qualitative, future work will focus on quantitative evaluation methods, such as using synthetic datasets with known ground-truth explanations. Additionally, the authors aim to explore heatmap truncation to improve focus and conduct comparative studies with other interpretability techniques like perturbation-based analysis.

Summary updated

BibTeX
@inproceedings{vinogradova2020towards,
  author = {Kira Vinogradova and Alexandr Dibrov and Gene Myers},
  title = {{Towards Interpretable Semantic Segmentation via Gradient-Weighted Class Activation Mapping (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7244},
}
Figure