Automatic Pancreatic Ductal Adenocarcinoma Detection in Whole Slide Images Using Deep Convolutional Neural Networks

Hao Fu, Weiming Mi, Boju Pan, Yucheng Guo, Junjie Li, Rongyan Xu, Jie Zheng, Chunli Zou, Tao Zhang, Zhiyong Liang, Junzhong Zou, Hao ZouFrontiers in Oncology 2021 · DOI

TL;DR

This paper introduces a novel deep learning framework for automatically detecting pancreatic ductal adenocarcinoma (PDAC) in whole slide images (WSIs). By leveraging convolutional neural networks (CNNs) for both patch-level classification and segmentation, the model achieves high accuracy, offering a promising tool to assist pathologists and improve diagnostic efficiency for this deadly cancer.

What Problem Does This Paper Address?

Pancreatic ductal adenocarcinoma (PDAC) is a particularly aggressive cancer with a very low survival rate, largely due to late diagnosis and challenges in accurate detection. While histopathology is the gold standard for diagnosis, manual analysis is time-consuming, requires significant expertise, and can suffer from low concordance among pathologists. Existing automated methods often rely on hand-crafted features, which can be problem-specific and lack versatility. The authors aim to address these limitations by developing a deep learning approach that can automatically and accurately detect PDAC in WSIs, reducing the burden on pathologists and potentially improving early detection.

Methods

The proposed framework employs a two-stage deep learning approach:

  • Patch-Level Classification: A CNN, specifically Google's Inception V3 architecture, is used to classify individual image patches as either cancerous or normal. This model was trained with a global average pooling layer, a fully connected layer, and a softmax layer for final classification. Adam optimizer and categorical cross-entropy loss were used during training.
  • WSI-Level Classification: For each WSI, a "malignant probability heatmap" is generated based on the patch-level classification results. This heatmap is then used to extract 36 statistical features (e.g., mean, variance, sum of probabilities). These features are fed into a Light Gradient Boosting Machine (LightGBM) model for the final WSI-level diagnosis. This two-stage approach is noted as not being fully end-to-end due to computational constraints.
  • Patch-Level Segmentation: To complement classification, a U-Net architecture was employed for segmenting cancerous regions within patches. This allows for a more granular understanding of where the cancer is located.

The authors also utilized Grad-CAM for visualizing the attention maps of the CNN, helping to understand which image regions contribute most to the model's predictions.

Datasets

The study utilized a dataset comprising:

  • Training/Validation/Testing Data: 60 normal and 171 cancerous pancreas tissue samples, digitized as high-resolution WSIs. These were labeled and confirmed by a senior pathologist.
  • Patch-Level Classification Dataset: Patches were sampled from 60 normal and 30 cancerous WSIs, categorized as positive (cancerous cells/lesions) or negative (normal cells/tissue).
  • Patch-Level Segmentation Dataset: Six WSIs were used, with PDAC areas annotated by a pathologist. Patches were then cut from these annotated regions.
  • Independent Verification Dataset: 52 WSIs labeled as carcinoma were downloaded from The Cancer Genome Atlas (TCGA) to test the model's portability and robustness.

Data augmentation techniques, including rotation, flipping, and reflection, were applied to the patches to mitigate the limited raw data and prevent overfitting.

Key Insights and Ideas

  • Deep Learning as a Versatile Alternative: The paper strongly advocates for deep learning over traditional feature-based methods for histopathological image analysis, highlighting its versatility and ability to learn complex patterns without manual feature engineering.
  • Two-Stage Approach for WSI Analysis: Combining patch-level CNN analysis with a WSI-level classifier (LightGBM) provides a robust method for analyzing large WSIs, balancing computational feasibility with diagnostic accuracy.
  • Visualizing Model Decisions: The use of Grad-CAM and heatmaps is crucial for making the "black box" of deep learning more interpretable. This visualization helps understand why the model makes certain predictions, which is vital for clinical trust and adoption.
  • Nuclear Features as Key Indicators: The visualization results suggest that the model primarily identifies PDAC based on nuclear features, such as shape, irregularity, and chromatin characteristics. This aligns with pathologist's diagnostic cues.

Key Results

  • Patch-Level Classification: Achieved an accuracy of 95.3% on the test dataset. The recall for cancerous patches was particularly high at 97.38%, indicating a strong ability to identify positive cases, which is critical for avoiding missed diagnoses.
  • WSI-Level Classification: Achieved an impressive 100% classification accuracy on the test dataset (12 normal and 35 cancerous WSIs).
  • Independent Verification: The model demonstrated good generalizability, achieving 90.38% accuracy on the independent TCGA dataset.
  • Patch-Level Segmentation: Achieved a dice coefficient of 0.8465 on the test data, indicating good performance in delineating cancerous regions.

Qualitatively, the heatmaps generated by the model effectively highlighted suspicious regions, serving as a potential pre-screening tool for pathologists.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that the patch-level segmentation model might sometimes make decisions based on color contrast, and the classification model can be influenced by irregular cell nuclei distributions, leading to potential false positives.
  • The two-stage architecture for WSI-level classification is not fully end-to-end, which is noted as a limitation due to computational constraints.
  • The model's reliance on nuclear features might struggle with well-differentiated adenocarcinomas where nuclear features are similar to chronic pancreatitis, or with fibroblasts that can mimic tumor cells.
  • Future Work:
    • The authors plan to analyze morphological features of PDAC more comprehensively, potentially combining histological patterns with nuclear features for improved accuracy, especially for challenging cases.
  • Extracting features from WSIs at multiple magnification levels is proposed to capture both fine-grained cellular details and broader tissue structures.
  • Developing an end-to-end model for WSI-level classification is suggested if higher computational resources become available.
  • The ultimate goal is to integrate this method into clinical practice as a pre-screening tool for PDAC diagnosis.

Summary updated

Abstract

Pancreatic ductal adenocarcinoma (PDAC) is one of the deadliest cancer types worldwide, with the lowest 5-year survival rate among all kinds of cancers. Histopathology image analysis is considered a gold standard for PDAC detection and diagnosis. However, the manual diagnosis used in current clinical practice is a tedious and time-consuming task and diagnosis concordance can be low. With the development of digital imaging and machine learning, several scholars have proposed PDAC analysis approaches based on feature extraction methods that rely on field knowledge. However, feature-based classification methods are applicable only to a specific problem and lack versatility, so that the deep-learning method is becoming a vital alternative to feature extraction. This paper proposes the first deep convolutional neural network architecture for classifying and segmenting pancreatic histopathological images on a relatively large WSI dataset. Our automatic patch-level approach achieved 95.3% classification accuracy and the WSI-level approach achieved 100%. Additionally, we visualized the classification and segmentation outcomes of histopathological images to determine which areas of an image are more important for PDAC identification. Experimental results demonstrate that our proposed model can effectively diagnose PDAC using histopathological images, which illustrates the potential of this practical application.
BibTeX
@inproceedings{fu2021automatic,
  author = {Hao Fu and Weiming Mi and Boju Pan and Yucheng Guo and Junjie Li and Rongyan Xu and Jie Zheng and Chunli Zou and Tao Zhang and Zhiyong Liang and Junzhong Zou and Hao Zou},
  title = {{Automatic Pancreatic Ductal Adenocarcinoma Detection in Whole Slide Images Using Deep Convolutional Neural Networks}},
  booktitle = {Frontiers in Oncology},
  year = {2021},
  doi = {10.3389/fonc.2021.665929},
}
Figure