Development and Validation of a Deep Learning Algorithm to Automatic Detection of Pituitary Microadenoma From MRI

Qingling Li, Yanhua Zhu, Minglin Chen, Ruomi Guo, Qingyong Hu, Yaxin Lu, Zhenghui Deng, Songqing Deng, Tiecheng Zhang, Huiquan Wen, Rong Gao, Yuanpeng Nie, Haicheng Li, Jianning Chen, Guojun Shi, Jun Shen, Wai Cheung, Zifeng Liu, Yulan Guo, Yanming ChenFrontiers in Medicine 2021 · DOI

TL;DR

This paper introduces a novel deep learning system (PM-CAD) for the automatic detection of pituitary microadenomas (PMs) from MRI scans. The system achieves high diagnostic accuracy, comparable to experienced radiologists, and demonstrates excellent generalization across different datasets, offering a promising tool to aid in the diagnosis of these challenging-to-detect tumors.

What Problem Does This Paper Address?

Pituitary microadenomas (PMs) are small tumors (less than 10 mm) that can cause significant health issues by secreting hormones. However, their small size and variable anatomy make them notoriously difficult to diagnose accurately using standard MRI interpretation alone. Manual analysis is time-consuming, prone to bias, and heavily reliant on the experience of radiologists, leading to potential delays in diagnosis and compromised patient care, especially in areas with a shortage of expert radiologists. This paper aims to bridge this diagnostic gap by developing an automated, AI-driven system to assist in PM detection.

Methods

The researchers developed a two-part deep learning system:

  1. Pituitary Region Detection: This module uses a Faster R-CNN model with a ResNet-50 FPN backbone to accurately identify and localize the pituitary gland within MRI images. This step ensures that subsequent analysis focuses on the relevant anatomical region.
  2. PM Diagnosis: A novel Convolutional Neural Network (CNN), named PM-CAD, is employed to diagnose the presence of PM within the cropped pituitary region. The PM-CAD model incorporates modifications to the ResNet architecture to preserve fine-grained features and includes an attention module to enhance feature representation. Techniques like histogram matching normalization, intensity shift data augmentation, and label-smoothing loss are used to combat overfitting and improve robustness.

The system was trained and validated using a large dataset of MRI scans.

Datasets

The study utilized a substantial dataset of 11,935 infertility participants. After exclusions, 1,520 participants (556 PM patients and 964 controls) were included. The data was partitioned into:

  • Development Set: 780 participants for model development.
  • Testing Set: 195 participants for initial performance evaluation.
  • Validation Set A: 545 participants, further divided into:
  • Validation Set A1 (Temporal Internal): Data from the same hospital but collected prospectively.
  • Validation Sets A2 & A3 (Geographical External): Data from two different hospitals, providing a robust test of generalization.
  • Validation Set B: 100 participants (50 PM, 50 controls) used for direct comparison with radiologists.
  • Validation Set C: Three clinically misdiagnosed PM cases to assess the system's performance on challenging scenarios.

Key Insights and Ideas

  • Deep Learning for Small Lesion Detection: The study demonstrates that deep learning models can be effectively trained to detect very small lesions like pituitary microadenomas, which are often challenging for human interpretation.
  • Two-Stage Approach for Robustness: The combination of a dedicated region detection module followed by a classification module likely contributes to the system's high accuracy by ensuring the correct anatomical area is analyzed and then classifying it.
  • Generalization is Key: The rigorous validation across temporal and geographical datasets is crucial. Achieving high accuracy on unseen data from different institutions validates the system's real-world applicability.
  • AI as a Radiologist's Assistant: The comparison with radiologists highlights that the PM-CAD system can serve as a powerful assistive tool, potentially reducing workload and improving diagnostic consistency, rather than a complete replacement.
  • Addressing Misdiagnosis: The system's ability to correctly diagnose cases that were previously misdiagnosed by radiologists underscores its potential to improve patient outcomes by preventing diagnostic delays.

Key Results

  • Testing Set Performance: The PM-CAD system achieved a diagnostic accuracy of 94.36% and an AUC of 98.13% on the testing dataset.
  • Validation Performance:
    • Internal validation (Validation set A1) showed an accuracy of 96.50% and an AUC of 95.46%.
  • External validation (Validation sets A2 & A3) demonstrated accuracies of 92.26% and 92.36%, with AUCs of 94.72% and 93.70%, respectively.
  • Human-Computer Competition: The PM-CAD system's performance was comparable to radiologists with over 10 years of experience, achieving similar accuracy (94.0% vs. 95.0%) and AUC (95.6% vs. 95.0%).
  • Misdiagnosis Cases: The system achieved 100% accuracy in diagnosing three cases that were previously misdiagnosed by radiologists.
  • Speed: The browser-based software application can analyze MR images in just 1-2 seconds.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that while the system achieves high accuracy, a small percentage of cases (3.5%) may still be misdiagnosed.
  • MRI data privacy concerns limit the ability to fine-tune the model for specific hospitals.
  • Future Work:
    • Collecting more data to further improve model performance.
  • Implementing a continual learning approach for model self-updates.
  • Exploring federated learning frameworks to enable privacy-preserving model fine-tuning across different institutions.

Summary updated

Abstract

<b>Background:</b> It is often difficult to diagnose pituitary microadenoma (PM) by MRI alone, due to its relatively small size, variable anatomical structure, complex clinical symptoms, and signs among individuals. We develop and validate a deep learning -based system to diagnose PM from MRI. <b>Methods:</b> A total of 11,935 infertility participants were initially recruited for this project. After applying the exclusion criteria, 1,520 participants (556 PM patients and 964 controls subjects) were included for further stratified into 3 non-overlapping cohorts. The data used for the training set were derived from a retrospective study, and in the validation dataset, prospective temporal and geographical validation set were adopted. A total of 780 participants were used for training, 195 participants for testing, and 545 participants were used to validate the diagnosis performance. The PM-computer-aided diagnosis (PM-CAD) system consists of two parts: pituitary region detection and PM diagnosis. The diagnosis performance of the PM-CAD system was measured using the receiver operating characteristics (ROC) curve and area under the ROC curve (AUC), calibration curve, accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1-score. <b>Results:</b> Pituitary microadenoma-computer-aided diagnosis system showed 94.36% diagnostic accuracy and 98.13% AUC score in the testing dataset. We confirm the robustness and generalization of our PM-CAD system, the diagnostic accuracy in the internal dataset was 96.50% and in the external dataset was 92.26 and 92.36%, the AUC was 95.5, 94.7, and 93.7%, respectively. In human-computer competition, the diagnosis performance of our PM-CAD system was comparable to radiologists with >10 years of professional expertise (diagnosis accuracy of 94.0% vs. 95.0%, AUC of 95.6% vs. 95.0%). For the misdiagnosis cases from radiologists, our system showed a 100% accurate diagnosis. A browser-based software was designed to assist the PM diagnosis. <b>Conclusions:</b> This is the first report showing that the PM-CAD system is a viable tool for detecting PM. Our results suggest that the PM-CAD system is applicable to radiology departments, especially in primary health care institutions.
BibTeX
@inproceedings{li2021development,
  author = {Qingling Li and Yanhua Zhu and Minglin Chen and Ruomi Guo and Qingyong Hu and Yaxin Lu and Zhenghui Deng and Songqing Deng and Tiecheng Zhang and Huiquan Wen and Rong Gao and Yuanpeng Nie and Haicheng Li and Jianning Chen and Guojun Shi and Jun Shen and Wai Cheung and Zifeng Liu and Yulan Guo and Yanming Chen},
  title = {{Development and Validation of a Deep Learning Algorithm to Automatic Detection of Pituitary Microadenoma From MRI}},
  booktitle = {Frontiers in Medicine},
  year = {2021},
  doi = {10.3389/fmed.2021.758690},
}
Figure