Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations

Guannan Zhao, Bo Zhou, Kaiwen Wang, Rui Jiang, Min Xu

TL;DR

Respond-CAM is a new visualization algorithm designed to make Convolutional Neural Networks (CNNs) interpretable, specifically for complex 3D biomedical imaging. By ensuring a "sum-to-score" property, it provides cleaner, more consistent heatmaps than the current state-of-the-art (Grad-CAM), helping researchers understand exactly which features in a 3D volume drive a model's prediction.

What Problem Does This Paper Address?

Deep learning has revolutionized biomedical image analysis, but CNNs remain "black boxes." In fields like structural biology, where researchers use Cellular Electron Cryo-Tomography (CECT) to look at macromolecular complexes, it isn't enough for a model to just classify a structure—scientists need to know why it made that choice.

Existing visualization tools like Grad-CAM were built for 2D natural images (like photos of cats and dogs). When applied to noisy, high-dimensional 3D biomedical data, these methods often produce "noisy" heatmaps or even highlight the wrong areas. The authors identified a lack of consistency between the visual explanation (the heatmap) and the actual numerical output of the model (the class score), a gap that makes these tools less reliable for sensitive scientific research.

Methods

The authors propose Respond-weighted Class Activation Mapping (Respond-CAM). Like its predecessor Grad-CAM, it uses the gradients of a target class flowing into the final convolutional layer to produce a heatmap.

However, Respond-CAM introduces a crucial change in how weights ($\alpha$) are calculated. Instead of a simple global average of gradients, Respond-CAM calculates a weighted average that incorporates the feature map activations themselves. This "responsiveness" factor measures how the class score changes as the feature map changes.

The mathematical "secret sauce" of Respond-CAM is the sum-to-score property. The authors prove that for a CNN with one fully-connected layer, the sum of all values in the Respond-CAM heatmap (plus a bias term) is exactly equal to the model’s prediction score. This ensures that the visualization is a direct, faithful decomposition of the model's decision-making process.

Datasets

The researchers evaluated their method on two primary types of data:

  1. 3D CECT Subtomograms: A simulated dataset of 23,000 3D volumes ($40^3$ voxels) representing 23 different structural classes of macromolecules. They tested both noise-free and high-noise versions.
  2. 2D Natural Images: To ensure the method generalizes, they used the PASCAL 2007 test set with a pre-trained VGG16 network to compare results against Grad-CAM on standard computer vision tasks.

Key Insights and Ideas

  • The Consistency Gap: The most significant insight is that a visualization tool is only as good as its consistency with the model's output. If a heatmap highlights a region but the sum of those highlights doesn't correlate with the prediction score, the "explanation" is misleading.
  • Smoothing via Weighting: By using the feature maps to weight the gradients (Equation 1 in the paper), Respond-CAM effectively "smoothens" the noise that often plagues gradient-based methods in 3D space.
  • Responsiveness vs. Importance: While Grad-CAM looks for "importance," Respond-CAM looks for "responsiveness"—how the model's output reacts to specific spatial features. This proves much more robust in the low signal-to-noise environments typical of cryo-electron microscopy.

Key Results

The paper demonstrates that Respond-CAM significantly outperforms Grad-CAM in 3D environments:

  • Visual Clarity: In 3D CECT data, Grad-CAM often produced negative values or highlighted background noise. Respond-CAM consistently highlighted the actual macromolecule of interest with high precision.
  • Quantitative Accuracy: Using the L1 error (the difference between the heatmap sum and the class score), Respond-CAM achieved scores near 0.5-1.0, while Grad-CAM lagged far behind with errors as high as 14.0-21.0.
  • Rank Correlation: Using Kendall’s Tau (KT) to measure how well the heatmap sums match the ranking of class predictions, Respond-CAM scored near-perfectly (~0.98), whereas Grad-CAM was much less consistent (as low as -0.11 in noisy data).
  • Natural Images: On 2D images (e.g., identifying a "Siamese cat" or "tricycle"), Respond-CAM produced results comparable to or slightly cleaner than Grad-CAM, proving it is a versatile tool for both 2D and 3D tasks.

Limitations and Future Work

The authors acknowledge that while the "sum-to-score" property is mathematically guaranteed for models with a single fully-connected (FC) layer, it becomes an approximation for deeper models with multiple FC layers due to non-linearities. While their experiments show the approximation holds up well in practice, a formal proof for deeper architectures remains an open challenge.

Future work could involve extending this logic to even more complex architectures like Transformers or applying it to real-time 3D segmentation tasks in clinical settings.

Summary updated

Abstract

The convolutional neural network (CNN) has become a powerful tool for various biomedical image analysis tasks, but there is a lack of visual explanation for the machinery of CNNs. In this paper, we present a novel algorithm, Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based models interpretable by visualizing input regions that are important for predictions, especially for biomedical 3D imaging data inputs. Our method uses the gradients of any target concept (e.g. the score of target class) that flow into a convolutional layer. The weighted feature maps are combined to produce a heatmap that highlights the important regions in the image for predicting the target concept. We prove a preferable sum-to-score property of the Respond-CAM and verify its significant improvement on 3D images from the current state-of-the-art approach. Our tests on Cellular Electron Cryo-Tomography 3D images show that Respond-CAM achieves superior performance on visualizing the CNNs with 3D biomedical image inputs, and is able to get reasonably good results on visualizing the CNNs with natural image inputs. The Respond-CAM is an efficient and reliable approach for visualizing the CNN machinery, and is applicable to a wide variety of CNN model families and image analysis tasks. Our code is available at: https://github.com/xulabs/projects/tree/master/respond_cam .
BibTeX
@inproceedings{zhao2018respond,
  author = {Guannan Zhao and Bo Zhou and Kaiwen Wang and Rui Jiang and Min Xu},
  title = {{Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2018},
  url = {https://openreview.net/forum?id=UxQ_M4LB9z},
}

Loading related content...

Figure