Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations

Guannan Zhao, Bo Zhou, Kaiwen Wang, Rui Jiang, Min Xu

Abstract

The convolutional neural network (CNN) has become a powerful tool for various biomedical image analysis tasks, but there is a lack of visual explanation for the machinery of CNNs. In this paper, we present a novel algorithm, Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based models interpretable by visualizing input regions that are important for predictions, especially for biomedical 3D imaging data inputs. Our method uses the gradients of any target concept that flows into a convolutional layer. The weighted feature maps are combined to produce a heatmap that highlights the important regions in the image for predicting the target concept. We prove a preferable sum-to-score property of the Respond-CAM and verify its significant improvement on 3D images from the current state-of-the-art approach.

Introduction: The Black Box of 3D Imaging

Deep learning has revolutionized biomedical image analysis, yet CNNs remain largely "black boxes." In structural biology, researchers utilize Cellular Electron Cryo-Tomography (CECT) to examine macromolecular complexes. For these scientists, it is insufficient for a model to merely classify a structure; they must understand the underlying features driving that classification.

Existing visualization tools like Grad-CAM were primarily developed for 2D natural images. When applied to high-dimensional, noisy 3D biomedical data, these methods often produce "noisy" heatmaps or highlight irrelevant areas. A critical issue identified by researchers is the lack of consistency between the visual explanation (the heatmap) and the numerical output of the model (the class score). This gap undermines the reliability of such tools in sensitive scientific research.

Methods: Respond-weighted Class Activation Mapping

We propose Respond-weighted Class Activation Mapping (Respond-CAM) to address these limitations. Similar to Grad-CAM, our method utilizes the gradients of a target class flowing into the final convolutional layer to generate a heatmap. However, Respond-CAM introduces a fundamental change in weight calculation.

The Respond-CAM workflow illustrating forward and backward propagation to generate weighted feature maps.

Instead of a simple global average of gradients, Respond-CAM calculates a weighted average that incorporates the feature map activations themselves. This "responsiveness" factor measures how the class score changes in relation to changes in the feature map.

The mathematical foundation of Respond-CAM is the sum-to-score property. We prove that for a CNN with one fully-connected layer, the sum of all values in the Respond-CAM heatmap (plus a bias term) is exactly equal to the model’s prediction score. This ensures the visualization is a faithful decomposition of the model's decision-making process.

Datasets and Evaluation

The method was evaluated on two distinct data types:

  1. 3D CECT Subtomograms: A simulated dataset of 23,000 3D volumes ($40^3$ voxels) representing 23 macromolecule classes, tested across varying noise levels.
  2. 2D Natural Images: The PASCAL 2007 test set with a pre-trained VGG16 network was used to ensure the method generalizes to standard computer vision tasks.

Key Results

Respond-CAM significantly outperforms Grad-CAM in 3D environments, particularly regarding visual clarity and quantitative consistency.

Comparison of Respond-CAM and Grad-CAM on 3D data with varying noise levels, showing Respond-CAM's superior localization of the object of interest.
  • Visual Clarity: In 3D CECT data, Grad-CAM often highlighted background noise or produced negative values. Respond-CAM consistently localized the actual macromolecule of interest with high precision.
  • Quantitative Accuracy: Using L1 error (the difference between the heatmap sum and the class score), Respond-CAM achieved scores between 0.5 and 1.0, whereas Grad-CAM's errors ranged from 14.0 to 21.0.
  • Rank Correlation: Measured by Kendall’s Tau (KT), Respond-CAM achieved near-perfect scores (~0.98), indicating that the heatmap sums accurately reflect the ranking of class predictions. In contrast, Grad-CAM showed poor consistency, dropping as low as -0.11 in noisy data.
  • Generalization: On 2D natural images, Respond-CAM produced results comparable to or cleaner than Grad-CAM, demonstrating its versatility.

Conclusion

Respond-CAM provides an efficient and reliable approach for visualizing CNN machinery. By ensuring the sum-to-score property, it bridges the gap between visual explanation and numerical prediction. While the property is an approximation for models with multiple fully-connected layers, empirical results demonstrate its robustness. Future work will explore extending this logic to more complex architectures like Transformers and real-time clinical 3D segmentation.

Summary updated

Abstract

The convolutional neural network (CNN) has become a powerful tool for various biomedical image analysis tasks, but there is a lack of visual explanation for the machinery of CNNs. In this paper, we present a novel algorithm, Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based models interpretable by visualizing input regions that are important for predictions, especially for biomedical 3D imaging data inputs. Our method uses the gradients of any target concept (e.g. the score of target class) that flows into a convolutional layer. The weighted feature maps are combined to produce a heatmap that highlights the important regions in the image for predicting the target concept. We prove a preferable sum-to-score property of the Respond-CAM and verify its significant improvement on 3D images from the current state-of-the-art approach. Our tests on Cellular Electron Cryo-Tomography 3D images show that Respond-CAM achieves superior performance on visualizing the CNNs with 3D biomedical images inputs, and is able to get reasonably good results on visualizing the CNNs with natural image inputs. The Respond-CAM is an efficient and reliable approach for visualizing the CNN machinery, and is applicable to a wide variety of CNN model families and image analysis tasks.
BibTeX
@inproceedings{zhao2018respond,
  author = {Guannan Zhao and Bo Zhou and Kaiwen Wang and Rui Jiang and Min Xu},
  title = {{Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2018},
  url = {https://openreview.net/forum?id=UxQ_M4LB9z},
}
Figure