PENet - a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

Shih-Cheng Huang, Tanay Kothari, Imon Banerjee, Chris Chute, Robyn L. Ball, Norah Borus, Andrew Huang, Bhavik N. Patel, Pranav Rajpurkar, Jeremy Irvin, Jared Dunnmon, Joseph Bledsoe, Katie S. Shpanskaya, Abhay Dhaliwal, Roham Zamanian, Andrew Y. Ng, Matthew P. LungrenNpj Digit. Medicine 2020 · OpenReview

TL;DR

This paper introduces PENet, a novel 77-layer 3D convolutional neural network (CNN) designed for the automated detection of pulmonary embolism (PE) from volumetric CT pulmonary angiography (CTPA) scans. PENet achieves strong diagnostic performance on both internal and external datasets, outperforming existing 3D CNN models and offering a scalable, end-to-end solution that could revolutionize PE diagnosis by prioritizing critical cases for radiologists.

What Problem Does This Paper Address?

Pulmonary embolism (PE) is a serious and often missed diagnosis, leading to significant morbidity and mortality. While CT pulmonary angiography (CTPA) is the gold standard for diagnosis, its interpretation is time-consuming and prone to errors due to factors like radiologist fatigue and inter-observer variability. The increasing volume of CTPA scans further strains healthcare systems. Existing automated PE detection methods often rely on complex, time-consuming preprocessing or manual feature engineering, limiting their clinical applicability and generalizability. This paper aims to bridge this gap by developing an end-to-end deep learning model that can directly process volumetric CTPA data for accurate and scalable PE detection.

Methods

The core of this work is PENet, a 77-layer 3D CNN. PENet's architecture is built upon several key components:

  • PENet Unit: Processes 3D input data using 3D convolutions, group normalization, and Leaky ReLU activation.
  • Squeeze-and-Excitation (SE) Block: Models inter-channel dependencies and adaptively recalibrates channel-wise features.
  • PENet Bottleneck: Combines multiple PENet units with SE blocks and skip connections.
  • PENet Encoder: Composed of multiple PENet bottlenecks stacked sequentially.

PENet was pretrained on the Kinetics-600 dataset (a common practice for leveraging large-scale video action recognition data for feature learning) and then fine-tuned on a retrospective CTPA dataset. A crucial aspect of their approach is the "end-to-end" nature, meaning it takes raw CTPA scans as input without requiring manual feature extraction or complex preprocessing steps like segmentation. For processing the large volumetric CTPA scans, PENet utilizes a sliding window approach, analyzing 24 consecutive slices at a time. This is justified by the observation that PE findings are rarely confined to a single slice, and this method helps to increase the relative proportion of the target PE within the input volume, improving computational efficiency and model optimization.

Datasets

  • Internal Dataset (Stanford): A retrospective dataset of 1797 CTPA studies from 1773 patients at Stanford University Medical Center. This dataset was split into training (1461 studies), validation (167 studies), and a hold-out test set (169 studies).
  • External Dataset (Intermountain): A dataset of 200 CTPA studies from 198 patients collected from Intermountain Healthcare. This dataset was used solely for evaluating the model's generalizability and was not used during training or validation.

The paper also mentions that subsegmental PE cases were excluded from the dataset, focusing on clinically significant PEs.

Key Insights and Ideas

  • End-to-End Learning for Complex 3D Data: The success of PENet demonstrates that deep learning models can effectively handle the complexity of volumetric CTPA data without manual feature engineering, which has been a significant hurdle in previous work.
  • Scalability and Generalizability: The model's strong performance on an external dataset from a different institution, using different scanner models and protocols, is a critical finding. This suggests PENet is robust and has the potential for widespread clinical adoption.
  • Sliding Window for Volumetric Data: The use of a 24-slice sliding window is an effective strategy for managing the large input size of CTPA scans while still capturing the necessary spatial context for PE detection.
  • Triage Tool Potential: The authors highlight PENet's potential as a triage tool. By automatically identifying potentially positive cases, it can prioritize studies for radiologists, leading to faster diagnoses for critical patients and potentially reducing the burden on radiology departments.
  • Interpretability with CAMs: The use of Class Activation Maps (CAMs) provides visual explanations for the model's predictions, allowing clinicians to understand why the model flagged a particular region, fostering trust and aiding in error analysis.

Key Results

  • Diagnostic Performance: PENet achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.84 [0.82-0.87] on the internal Stanford test set and 0.85 [0.81-0.88] on the external Intermountain test set.
  • Outperformance: PENet outperformed other state-of-the-art 3D CNN models (ResNet3D, ResNeXt3D, DenseNet3D) by a margin of 0.04 AUROC on the internal test set and 0.02 AUROC on the external test set.
  • Sensitivity vs. Specificity Tuning: The model can be tuned to achieve high sensitivity (e.g., 0.91) at the cost of specificity (e.g., 0.43), making it suitable for a triage scenario where missing a PE is more critical than having some false positives.
  • Pretraining Benefits: Pretraining PENet on the Kinetics-600 dataset significantly improved its performance, increasing AUROC by 0.15 on the internal and 0.23 on the external test set compared to a non-pretrained model.

Limitations and Future Work

  • Retrospective Design: The study is retrospective, and while performance was sustained on an external dataset, universal generalizable performance is not yet confirmed.
  • Focus on Clinically Important PE: The model was optimized for clinically important PEs, excluding subsegmental cases. This means it may not detect all types of PE.
  • Limited to PE: CTPA scans are often used to diagnose other conditions. PENet is currently specialized for PE detection and does not analyze for other pathologies.
  • Artifacts Not Explicitly Addressed: The paper acknowledges that specific artifacts were not explicitly analyzed, though the model was trained on data that included them.
  • Future Work: The authors suggest further studies are needed to evaluate the optimal integration of PENet and similar models into clinical workflows. They also plan a full open-source release of the data and trained model, which will be crucial for reproducibility and further research.

Summary updated

Abstract

Pulmonary embolism (PE) is a life-threatening clinical problem and computed tomography pulmonary angiography (CTPA) is the gold standard for diagnosis. Prompt diagnosis and immediate treatment are critical to avoid high morbidity and mortality rates, yet PE remains among the diagnoses most frequently missed or delayed. In this study, we developed a deep learning model—PENet, to automatically detect PE on volumetric CTPA scans as an end-to-end solution for this purpose. The PENet is a 77-layer 3D convolutional neural network (CNN) pretrained on the Kinetics-600 dataset and fine-tuned on a retrospective CTPA dataset collected from a single academic institution. The PENet model performance was evaluated in detecting PE on data from two different institutions: one as a hold-out dataset from the same institution as the training data and a second collected from an external institution to evaluate model generalizability to an unrelated population dataset. PENet achieved an AUROC of 0.84 [0.82–0.87] on detecting PE on the hold out internal test set and 0.85 [0.81–0.88] on external dataset. PENet also outperformed current state-of-the-art 3D CNN models. The results represent successful application of an end-to-end 3D CNN model for the complex task of PE diagnosis without requiring computationally intensive and time consuming preprocessing and demonstrates sustained performance on data from an external institution. Our model could be applied as a triage tool to automatically identify clinically important PEs allowing for prioritization for diagnostic radiology interpretation and improved care pathways via more efficient diagnosis.
BibTeX
@inproceedings{huang2020penet,
  author = {Shih-Cheng Huang and Tanay Kothari and Imon Banerjee and Chris Chute and Robyn L. Ball and Norah Borus and Andrew Huang and Bhavik N. Patel and Pranav Rajpurkar and Jeremy Irvin and Jared Dunnmon and Joseph Bledsoe and Katie S. Shpanskaya and Abhay Dhaliwal and Roham Zamanian and Andrew Y. Ng and Matthew P. Lungren},
  title = {{PENet - a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging}},
  booktitle = {Npj Digit. Medicine},
  year = {2020},
  url = {https://openreview.net/forum?id=fRU5xlLSPh},
}
Figure