Automatic Prostate Zonal Segmentation Using Fully Convolutional Network With Feature Pyramid Attention

Yongkai Liu, Kyung Hyun Sung, Guang Yang, Sohrab Afshari Mirak, Melina Hosseiny, Afshin Azadikhah, Xinran Zhong, Robert E. Reiter, Yeejin Lee, Steven S. Raman

TL;DR

This paper introduces a novel deep learning model, a Fully Convolutional Network with Feature Pyramid Attention, for automatically segmenting prostate zones (transition zone - TZ and peripheral zone - PZ) in T2-weighted MRI scans. The model achieves high accuracy, comparable to human experts, and demonstrates consistent performance across internal and external datasets, paving the way for more reproducible prostate cancer analysis.

What Problem Does This Paper Address?

Prostate cancer (PCa) diagnosis and staging heavily rely on multiparametric MRI (mpMRI), particularly T2-weighted (T2w) images. The Prostate Imaging Reporting and Data System (PI-RADSv2.1) uses T2w and diffusion-weighted imaging to assess lesions in specific prostate zones (TZ and PZ). However, manually segmenting these zones is time-consuming, subjective, and prone to inter-reader variability, hindering consistent and reproducible analysis. This variability can impact the accuracy of lesion localization and subsequent risk stratification. The paper aims to address this by developing an automated, accurate, and reliable method for prostate zonal segmentation.

Methods

The core of the proposed solution is a novel deep learning architecture based on a Fully Convolutional Network (FCN). This network is composed of three main components:

  • Improved ResNet50 Encoder: This module is designed to capture rich semantic information from the T2w MRI slices. The authors modified the standard ResNet50 by removing the initial max pooling layer and adjusting the blocks in the fourth layer to include dilated convolutions. This aims to preserve spatial information while extracting deeper features.
  • Feature Pyramid Attention Network: This novel component is crucial for capturing contextual information at multiple scales. It uses pyramid structures with different kernel sizes (3x3, 5x5, 7x7) to extract features at various resolutions. An attention mechanism then integrates these multi-scale features, allowing the model to focus on relevant details and global context.
  • Decoder Network: This part of the architecture upsamples the features to recover the original image dimensions, preparing them for the final segmentation output.

The entire network is trained end-to-end using a cross-entropy loss function. Data augmentation techniques like horizontal flipping, random rotation, and elastic transformations were employed to enhance the model's robustness.

Datasets

The study utilized two distinct datasets for training and evaluation:

  • PROSTATEX Challenge Dataset: This publicly available dataset was used for model development and internal testing. It comprised 313 scans for development and an additional 63 scans for internal testing (ITD).
  • External Testing Dataset (ETD): This dataset was collected from a large U.S. tertiary academic medical center and consisted of 46 MRI scans. This dataset was used to evaluate the model's generalization capabilities on unseen data.

Manual annotations for the TZ and PZ contours were performed by research fellows and supervised by expert genitourinary (GU) radiologists for both datasets.

Key Insights and Ideas

  • Feature Pyramid Attention is Key: The integration of a feature pyramid attention mechanism significantly enhances the model's ability to capture multi-scale contextual information, which is vital for accurately delineating the complex boundaries of prostate zones.
  • ResNet50 Modifications Improve Performance: Adapting the ResNet50 encoder by removing max pooling and incorporating dilated convolutions helps preserve crucial spatial details, leading to better segmentation results compared to a standard ResNet50.
  • Consistency Across Datasets is Achieved: The model demonstrates remarkably similar performance on both the internal (PROSTATEX) and external (tertiary medical center) datasets, indicating strong generalization capabilities. This is particularly important for real-world clinical applications.
  • Model Performance Rivals Human Experts: The quantitative results show that the proposed model's segmentation accuracy (measured by Dice Similarity Coefficient - DSC) is comparable to, and in some cases exceeds, the inter-reader consistency between two expert radiologists on the external testing dataset.
  • PZ Segmentation is Inherently More Challenging: The study highlights that manual segmentation of the peripheral zone (PZ) exhibits lower inter-reader consistency than the transition zone (TZ). This suggests that the PZ's complex shape and less distinct boundaries pose a greater challenge for both automated and manual segmentation methods.

Key Results

  • Internal Testing Dataset (ITD): The proposed method achieved mean DSCs of 0.74±0.08 for PZ and 0.86±0.07 for TZ.
  • External Testing Dataset (ETD): The model achieved mean DSCs of 0.74±0.07 for PZ and 0.79±0.12 for TZ.
  • Comparison with U-Net: The proposed method significantly outperformed the baseline U-Net on the ITD for both PZ and TZ segmentation.
  • Inter-Reader Consistency: The inter-reader consistency between two experts was 0.71±0.13 for PZ and 0.75±0.14 for TZ. The proposed model's performance was comparable to or better than this consistency.
  • Generalization: The performance difference between ITD and ETD was minimal for PZ (no significant difference) and less than 10% for TZ, demonstrating good generalization.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Single MRI Vendor: The study primarily used MRI data from a single vendor (Siemens Healthineers). Future work should incorporate datasets from different vendors and with varying in-plane resolutions to further validate the model's robustness.
  • 2D FCN Model: The current model is a 2D FCN, which doesn't fully leverage inter-slice correlation information. Future research will explore improving the model to capture this 3D context, potentially leading to even more accurate segmentation.
  • Limited Number of Experts: The inter-reader consistency was assessed with only two experts. Including more experts in future studies would provide a more robust measure of inter-reader agreement.
  • Bounding Box Pre-processing: The authors suggest that incorporating a bounding box approach for prostate localization as a pre-processing step, similar to other studies, could further improve segmentation performance and reduce false positives.

Summary updated

Abstract

Our main objective in the paper is to develop a novel deep learning-based algorithm for automatic segmentation of prostate zones and to evaluate the performance of the algorithm on an additional independent testing dataset in comparison with inter-reader agreement between two experts. With IRB approval and HIPAA compliance, we designed a novel convolutional neural network (CNN) for automatic segmentation of the prostatic transition zone (TZ) and peripheral zone (PZ) on T2-weighted (T2w) 3 Tesla (3T) MRI. The total study cohort included 359 MRI scans of patients in subcohorts; 313 scans from a deidentified publicly available dataset (SPIE-AAPM-NCI PROSTATEX challenge) and 46 scans from a large U.S. tertiary referral center (external testing dataset (ETD)). The TZ and PZ contours were manually annotated by research fellows, supervised by expert genitourinary (GU) radiologists. The model was developed using 250 patients and tested internally using the remaining 63 patients from the PROSTATEX (internal testing dataset (ITD)) and tested again (n=46) externally using the ETD. The Dice Similarity Coefficient (DSC) was used to evaluate the segmentation performance. DSCs for PZ and TZ were 0.74±0.08 and 0.86±0.07 in the ITD respectively. In the ETD, DSCs for PZ and TZ were 0.74±0.07 and 0.79±0.12, respectively. The inter-reader consistency (Expert 2 vs. Expert 1) were 0.71±0.13 (PZ) and 0.75±0.14 (TZ). This novel DL algorithm enabled automatic segmentation of PZ and TZ with high accuracy on both ITD and ETD without a performance difference for PZ and less than 10% TZ difference. In the ETD, the proposed method can be comparable to experts in the segmentation of prostate zones. Part of our source code and datasets with annotations is available at https://github.com/ykl-ucla/prostate_zonal_seg.
BibTeX
@inproceedings{liu2019automatic,
  author = {Yongkai Liu and Kyung Hyun Sung and Guang Yang and Sohrab Afshari Mirak and Melina Hosseiny and Afshin Azadikhah and Xinran Zhong and Robert E. Reiter and Yeejin Lee and Steven S. Raman},
  title = {{Automatic Prostate Zonal Segmentation Using Fully Convolutional Network With Feature Pyramid Attention}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=nd7pr7AVu8},
}
Figure