SISC: End-to-End Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells

Devinder Kumar, Vignesh Sankar, David A. Clausi, Graham W. Taylor, Alexander Wong

TL;DR

This paper introduces a novel deep learning framework called SISC (Stacked Interpretable Sequencing Cells) for end-to-end lung cancer prediction from CT scans. SISC not only achieves state-of-the-art prediction accuracy but also provides interpretable "critical response maps" that highlight the regions in the CT scan most influential for its predictions, fostering trust and collaboration between radiologists and AI.

What Problem Does This Paper Address?

Lung cancer remains a leading cause of cancer-related deaths globally, making early and accurate diagnosis crucial. While computer-aided diagnosis (CAD) systems using CT scans show promise, traditional methods often rely on hand-engineered "radiomic features." These features, though informative, can be limited in their ability to capture the full complexity of tumor characteristics. More recent "discovery radiomics" approaches learn these features directly from data, achieving superior performance. However, a significant challenge with these powerful deep learning models is their "black box" nature, making it difficult for clinicians to understand why a prediction is made, hindering trust and adoption. This paper aims to bridge this gap by developing a discovery radiomics system that is both highly accurate and interpretable.

Methods

The core innovation is the Stacked Interpretable Sequencing Cells (SISC) architecture. This framework builds a radiomic sequencer by stacking multiple "interpretable sequencing cells" in a depth-wise manner. Each cell is a modular block of convolutional layers, batch normalization, dropout, and ReLU activations. This modular design simplifies the search for optimal architectures and improves predictability.

Crucially, the SISC architecture is designed for end-to-end interpretability. After the prediction is made, a process involving backpropagation through the SISC layers generates critical response maps. These maps act as heatmaps, highlighting the specific regions within the CT scan that contributed most to the model's decision for both benign and malignant classifications. This allows clinicians to visually inspect the evidence supporting the AI's diagnosis.

Datasets

The primary dataset used is the Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI). This dataset contains thoracic CT scans with annotations from multiple radiologists, providing ground truth for nodule characteristics and malignancy scores. The authors specifically focused on nodules larger than 3mm, resulting in a dataset of 2669 nodules. They also created three distinct datasets ('M', 'B', and 'I') by treating malignancy score 3 differently (as malignant, benign, or ignored) to account for varying interpretations. Data augmentation techniques were employed to address class imbalance and enhance model generalizability.

Key Insights and Ideas

  • Interpretability as a Design Goal: Instead of treating interpretability as an afterthought, the SISC architecture is designed from the ground up to facilitate it. This is achieved by making the sequencing cells themselves interpretable and using their stacked nature to generate response maps.
  • Critical Response Maps as a Bridge: The generated critical response maps are a key contribution. They offer a visual explanation for the AI's prediction, allowing radiologists to validate the model's reasoning and potentially identify subtle features they might have missed.
  • Modular Design for Efficiency: The use of interpretable sequencing cells as building blocks simplifies the architecture design process and allows for more predictable optimization.
  • Discovery Radiomics with Transparency: The paper demonstrates that achieving state-of-the-art performance in discovery radiomics doesn't necessitate a completely opaque model. The SISC framework shows that high accuracy and interpretability can coexist.

Key Results

The SISC framework achieved state-of-the-art performance in lung cancer prediction across various metrics (accuracy, sensitivity, specificity, AUC) on the LIDC dataset. While specific numbers vary across the three experimental datasets ('M', 'B', 'I'), the paper shows comparable or superior results to existing methods, including those by Xie et al. [10]. For instance, on the 'M' dataset, the SISC achieved an AUC of 89.06±1.20, outperforming several prior methods. The critical response maps were shown to effectively highlight nodule regions and clinically relevant markers, demonstrating their utility in providing evidence for predictions.

Limitations and Future Work

  • Dataset Size and Augmentation: While data augmentation was used, the paper notes that datasets 'B' and 'M' were not initially evenly distributed. The performance was shown to be sensitive to dataset size, with approximately 15k samples yielding the best results for certain metrics.
  • Inter-Observer Variability: The paper acknowledges that radiologist annotations can have inter-observer variability, which is inherent in the dataset and can influence model training and evaluation.
  • Future Work: The authors suggest that the critical response maps can improve radiologist-machine collaboration, leading to more effective diagnosis. Further exploration of how these maps can be integrated into clinical workflows and potentially used for other medical imaging tasks would be valuable.

Summary updated

Abstract

Lung cancer is the leading cause of cancer-related death worldwide. Computer-aided diagnosis (CAD) systems have shown significant promise in recent years for facilitating the effective detection and classification of abnormal lung nodules in computed tomography (CT) scans. While hand-engineered radiomic features have been traditionally used for lung cancer prediction, there have been significant recent successes achieving state-of-the-art results in the area of discovery radiomics. Here, radiomic sequencers comprising of highly discriminative radiomic features are discovered directly from archival medical data. However, the interpretation of predictions made using such radiomic sequencers remains a challenge. A novel end-to-end interpretable discovery radiomics-driven lung cancer prediction pipeline has been designed, build, and tested. The radiomic sequencer being discovered possesses a deep architecture comprised of stacked interpretable sequencing cells (SISC). The SISC architecture is shown to outperform previous approaches while providing more insight in to its decision making process. The SISC radiomic sequencer is able to achieve state-of-the-art results in lung cancer prediction, and also offers prediction interpretability in the form of critical response maps. The critical response maps are useful for not only validating the predictions of the proposed SISC radiomic sequencer, but also provide improved radiologist-machine collaboration for effective diagnosis.
BibTeX
@article{kumar2019sisc,
  author = {Devinder Kumar and Vignesh Sankar and David A. Clausi and Graham W. Taylor and Alexander Wong},
  title = {{SISC: End-to-End Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2945524},
}
Figure