Deep Convolutional Neural Network-Assisted Feature Extraction for Diagnostic Discrimination and Feature Visualization in Pancreatic Ductal Adenocarcinoma (PDAC) versus Autoimmune Pancreatitis (AIP)

Sebastian Ziegelmayer, Georgios Kaissis, F Harder, Friederike Jungmann, Tamara Müller, Marcus R. Makowski, Rickmer BrarenJournal of Clinical Medicine 2020 · DOI

TL;DR

This paper introduces a deep learning approach to differentiate between autoimmune pancreatitis (AIP) and pancreatic ductal adenocarcinoma (PDAC) using CT scans. By leveraging deep features extracted from a Convolutional Neural Network (CNN), the model significantly outperforms traditional radiomic features in diagnostic accuracy, offering a promising avenue for improved non-invasive diagnosis.

What Problem Does This Paper Address?

Differentiating between autoimmune pancreatitis (AIP) and pancreatic ductal adenocarcinoma (PDAC) is a significant clinical challenge. These conditions often present with similar symptoms and imaging characteristics, leading to a risk of misdiagnosis. Misdiagnosis can have severe consequences, including unnecessary surgeries for AIP or delayed treatment for PDAC. While radiomics has shown promise in this differentiation, its features are limited to low-level image representations. This paper aims to address this by exploring the use of deep learning, which can extract more complex and abstract features from medical images, potentially leading to more accurate diagnostic models.

Methods

The study employed a retrospective approach, analyzing CT scans from patients diagnosed with either AIP or PDAC. The core methodology involved two main feature extraction techniques:

  • Traditional Radiomic Features: 1411 radiomic features were extracted from the pancreas using the PyRadiomics Python package. These features were then preprocessed to remove unstable and non-informative ones, resulting in 299 features.
  • Deep Features: A pre-trained VGG19 CNN, originally trained on the ImageNet dataset, was utilized as a feature extractor. CT image stacks were adapted to the CNN's input requirements, and 256 feature maps were extracted from an intermediate layer. These feature maps were then processed to yield 79 deep features.

Both sets of features underwent selection and normalization. An extremely randomized trees classifier was then trained and evaluated using a two-fold shuffle-split cross-validation strategy to discriminate between AIP and PDAC. Feature maps were also visualized to understand the model's decision-making process.

Datasets

The study utilized a dataset of 86 patients: 44 with AIP and 42 with PDAC. These CT scans were acquired during the portal venous phase. The dataset was collected retrospectively from the hospital picture archiving system (PACS) between November 2004 and March 2020. No new benchmark datasets were introduced; the focus was on evaluating the proposed method on an existing clinical cohort.

Key Insights and Ideas

  • Deep features capture richer information: The study strongly suggests that CNN-derived deep features are more adept at capturing complex patterns within medical images compared to traditional radiomic features, leading to superior diagnostic performance.
  • Visualizing feature maps aids interpretability: The ability to visualize feature maps from the CNN provides insights into where the model is "looking" to make its decisions. This visualization revealed distinct activation patterns for AIP and PDAC, suggesting that the CNN learns discriminative features related to lesion architecture.
  • Transfer learning is effective: Using a CNN pre-trained on a large natural image dataset (ImageNet) as a feature extractor proved effective for medical image analysis, even with a relatively small clinical dataset. This highlights the power of transfer learning in overcoming data limitations.
  • Age is not a primary differentiator: The analysis indicated that age, while statistically different between the groups, did not significantly improve the diagnostic performance when included as an additional feature, suggesting that the imaging features themselves are more critical for differentiation.

Key Results

The machine learning model trained on deep features achieved significantly better performance than the model trained on radiomic features:

  • Deep Features:
    • Sensitivity: 0.89 ± 0.11
    • Specificity: 0.83 ± 0.06
    • ROC-AUC: 0.90 ± 0.02
  • Radiomic Features:
    • Sensitivity: 0.72 ± 0.11
    • Specificity: 0.78 ± 0.06
    • ROC-AUC: 0.80 ± 0.01

The ROC curves visually confirmed the superior performance of the deep feature-based model. Feature map visualization showed that PDAC lesions had higher activation at the transition zone with normal tissue, while AIP lesions showed activation within the inner structure.

Limitations and Future Work

The authors acknowledge several limitations:

  • Retrospective nature and small cohort: The study's retrospective design and the relatively small patient cohort (86 patients) limit the generalizability of the findings.
  • Lack of external validation: The absence of an external validation set means the results might be overly optimistic due to cross-validation.
  • Transfer learning dependency: The reliance on transfer learning for deep feature extraction is a consequence of the limited dataset size.

Future work should focus on:

  • Multi-center prospective studies: To validate the findings and improve generalizability.
  • Investigating the benefit of CNNs and deep features on larger cohorts: To further confirm their diagnostic utility.
  • Exploring the impact of feature map visualization on radiological decision-making: To understand how these insights can be integrated into clinical practice.

Summary updated

Abstract

The differentiation of autoimmune pancreatitis (AIP) and pancreatic ductal adenocarcinoma (PDAC) poses a relevant diagnostic challenge and can lead to misdiagnosis and consequently poor patient outcome. Recent studies have shown that radiomics-based models can achieve high sensitivity and specificity in predicting both entities. However, radiomic features can only capture low level representations of the input image. In contrast, convolutional neural networks (CNNs) can learn and extract more complex representations which have been used for image classification to great success. In our retrospective observational study, we performed a deep learning-based feature extraction using CT-scans of both entities and compared the predictive value against traditional radiomic features. In total, 86 patients, 44 with AIP and 42 with PDACs, were analyzed. Whole pancreas segmentation was automatically performed on CT-scans during the portal venous phase. The segmentation masks were manually checked and corrected if necessary. In total, 1411 radiomic features were extracted using PyRadiomics and 256 features (deep features) were extracted using an intermediate layer of a convolutional neural network (CNN). After feature selection and normalization, an extremely randomized trees algorithm was trained and tested using a two-fold shuffle-split cross-validation with a test sample of 20% (<i>n</i> = 18) to discriminate between AIP or PDAC. Feature maps were plotted and visual difference was noted. The machine learning (ML) model achieved a sensitivity, specificity, and ROC-AUC of 0.89 ± 0.11, 0.83 ± 0.06, and 0.90 ± 0.02 for the deep features and 0.72 ± 0.11, 0.78 ± 0.06, and 0.80 ± 0.01 for the radiomic features. Visualization of feature maps indicated different activation patterns for AIP and PDAC. We successfully trained a machine learning model using deep feature extraction from CT-images to differentiate between AIP and PDAC. In comparison to traditional radiomic features, deep features achieved a higher sensitivity, specificity, and ROC-AUC. Visualization of deep features could further improve the diagnostic accuracy of non-invasive differentiation of AIP and PDAC.
BibTeX
@article{ziegelmayer2020deep,
  author = {Sebastian Ziegelmayer and Georgios Kaissis and F Harder and Friederike Jungmann and Tamara Müller and Marcus R. Makowski and Rickmer Braren},
  title = {{Deep Convolutional Neural Network-Assisted Feature Extraction for Diagnostic Discrimination and Feature Visualization in Pancreatic Ductal Adenocarcinoma (PDAC) versus Autoimmune Pancreatitis (AIP)}},
  journal = {Journal of Clinical Medicine},
  year = {2020},
  doi = {10.3390/jcm9124013},
}
Figure