Soft Activation Mapping of Lung Nodules in Low-Dose CT images

Yiming Lei, Yukun Tian, Hongming Shan, Junping Zhang, Ge Wang, Mannudeep K. Kalra

Abstract

A number of studies on lung nodule classification lack clinical/biological interpretations of the features extracted by convolutional neural network (CNN). The methods like class activation mapping (CAM) and gradient-based CAM (Grad-CAM) are tailored for interpreting localization and classification tasks while they ignored fine-grained features. Therefore, CAM and Grad-CAM cannot provide optimal interpretation for lung nodule categorization task in low-dose CT images, in that fine-grained pathological clues like discrete and irregular shape and margins of nodules are capable of enhancing sensitivity and specificity of nodule classification with regards to CNN. In this paper, we first develop a soft activation mapping (SAM) to enable fine-grained lung nodule shape & margin (LNSM) feature analysis with a CNN so that it can access rich discrete features. Secondly, by combining high-level convolutional features with SAM, we further propose a high-level feature enhancement scheme (HESAM) to localize LNSM features. Experiments on the LIDC-IDRI dataset indicate that 1) SAM captures more fine-grained and discrete attention regions than existing methods, 2) HESAM localizes more accurately on LNSM features and obtains the state-of-the-art predictive performance, reducing the false positive rate, and 3) we design and conduct a visually matching experiment which incorporates radiologists study to increase the confidence level of applying our method to clinical diagnosis.

Introduction

Deep learning has revolutionized medical imaging, but the "black box" nature of these models remains a significant barrier to clinical adoption. In lung cancer screening, radiologists rely on specific morphological indicators known as Lung Nodule Shape and Margin (LNSM) features. For instance, smooth, spherical nodules are typically benign, whereas nodules with "spiculated" (spiky) or irregular margins are high-risk indicators for malignancy.

Existing interpretability tools like Class Activation Mapping (CAM) and Grad-CAM often fail in this context. Designed for natural images, they tend to highlight the most salient central part of an object. For lung nodules, however, the most critical diagnostic information often resides at the boundaries. This paper introduces Soft Activation Mapping (SAM) and its enhanced version, HESAM, to bridge the gap between AI attention and clinical relevance by focusing on these fine-grained pathological clues.

Methodology

The proposed framework introduces two primary architectural innovations to improve both classification performance and interpretability:

  1. Soft Activation Mapping (SAM): Traditional CAM uses Global Average Pooling (GAP) to collapse feature maps, which often loses local spatial detail. SAM instead preserves local information by using a combination of average pooling and dedicated fully connected (FC) layers for each feature map. This allows the model to "discretize" features, enabling it to attend to multiple, small, disconnected regions—such as the tiny spikes on a nodule's margin—rather than a single central blob.
  2. High-level Enhanced SAM (HESAM): While SAM excels at capturing edge details, it can lose the broader structural context. HESAM addresses this by combining fine-grained SAM features with high-level structural features extracted from a UNet-style encoder-decoder architecture. This hybrid approach ensures the model understands both the overall nodule structure and its minute boundary details.
The overall HESAM architecture integrates a UNet-style encoder-decoder with SAM modules to capture both global structure and fine-grained margin features.

To further enhance performance, the model utilizes a multi-slice input strategy. By using 11 adjacent CT slices as input channels, the network gains a pseudo-3D perspective that improves feature extraction without the computational overhead or overfitting risks associated with full 3D CNNs.

Results and Discussion

The study utilized the LIDC-IDRI dataset, comprising nodules from 1,010 patients. The HESAM model achieved a state-of-the-art accuracy of 99.13% on the 11-channel dataset. Notably, it achieved a specificity of 0.9921, indicating a significant reduction in false positives—a critical metric for reducing unnecessary biopsies in clinical practice.

Beyond raw metrics, the qualitative results demonstrate that HESAM's attention maps align closely with radiologist observations. In visually matching experiments, HESAM consistently highlighted the irregular margins and spiculation that doctors use to diagnose malignancy, whereas traditional CAM and Grad-CAM maps were often biased toward the center of the nodule.

Visual comparison of attention maps: SAM and HESAM (bottom rows) capture more discrete, fine-grained features at the nodule margins compared to traditional CAM and Grad-CAM (middle rows).

Conclusion

By shifting the focus of CNN attention from global saliency to fine-grained shape and margin features, SAM and HESAM provide a more clinically grounded approach to lung nodule classification. The model not only achieves superior predictive performance but also offers transparent "reasoning" that matches the diagnostic patterns of human experts. Future work will explore extending this fine-grained analysis to automated lesion detection across entire 3D CT volumes.

Summary updated

Abstract

A number of studies on lung nodule classification lack clinical/biological interpretations of the features extracted by convolutional neural network (CNN). The methods like class activation mapping (CAM) and gradient-based CAM (Grad-CAM) are tailored for interpreting localization and classification tasks while they ignored fine-grained features. Therefore, CAM and Grad-CAM cannot provide optimal interpretation for lung nodule categorization task in low-dose CT images, in that fine-grained pathological clues like discrete and irregular shape and margins of nodules are capable of enhancing sensitivity and specificity of nodule classification with regards to CNN. In this paper, we first develop a soft activation mapping (SAM) to enable fine-grained lung nodule shape \& margin (LNSM) feature analysis with a CNN so that it can access rich discrete features. Secondly, by combining high-level convolutional features with SAM, we further propose a high-level feature enhancement scheme (HESAM) to localize LNSM features. Experiments on the LIDC-IDRI dataset indicate that 1) SAM captures more fine-grained and discrete attention regions than existing methods, 2) HESAM localizes more accurately on LNSM features and obtains the state-of-the-art predictive performance, reducing the false positive rate, and 3) we design and conduct a visually matching experiment which incorporates radiologists study to increase the confidence level of applying our method to clinical diagnosis.
BibTeX
@article{lei2020soft,
  author = {Yiming Lei and Yukun Tian and Hongming Shan and Junping Zhang and Ge Wang and Mannudeep K. Kalra},
  title = {{Soft Activation Mapping of Lung Nodules in Low-Dose CT images}},
  journal = {Medical Image Analysis},
  year = {2020},
  doi = {10.1016/j.media.2019.101628},
}
Figure