Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation

Jun Li, Xiaozhu Lin, Hui Che, Hao Li, Xiaohua QianarXiv 2019 · DOI · OpenReview

TL;DR

This paper introduces the Probability Map Guided Bi-directional Recurrent UNet (PBR-UNet), a novel deep learning architecture for pancreas segmentation in medical images. By fusing intra-slice information with inter-slice probabilistic maps, PBR-UNet effectively captures local 3D context, leading to more accurate and robust segmentation, especially for challenging cases, while maintaining computational efficiency.

What Problem Does This Paper Address?

Accurate segmentation of the pancreas in medical imaging (like CT scans) is crucial for diagnosis and treatment planning. However, this task is notoriously difficult due to significant variations in pancreas shape and volume across individuals. Traditional 2D segmentation methods struggle with capturing the full 3D context, leading to a loss of spatial information. On the other hand, full 3D segmentation methods, while capturing context, are computationally expensive and require substantial memory resources. This paper aims to bridge this gap by developing a method that leverages local 3D contextual information efficiently, overcoming the limitations of both 2D and 3D approaches.

Methods

The proposed PBR-UNet architecture is a two-stage approach:

  1. Initial Estimation Module: This module uses a multi-view FCN (M-FCN) to generate pixel-wise probabilistic maps for each slice. These maps indicate the likelihood of a voxel belonging to the pancreas.
  2. Primary Segmentation Module: This is the core of the innovation. It employs a 2.5D UNet architecture that incorporates a novel local 3D hybrid regularization scheme and a bi-directional recurrent optimization scheme.
  3. Local 3D Hybrid Information: The input image of a central slice is combined with the probabilistic maps of its adjacent slices. This creates a "hybrid multi-channel data" that encodes local 3D context.
  4. Bi-directional Recurrent Optimization: This scheme iteratively updates the hybrid information. It processes the data in both forward and backward directions, allowing the network to learn from both preceding and succeeding slices. This ensures that the segmentation of a given slice is informed by its full local 3D neighborhood, leading to more stable and accurate results. The process involves segmentation (using the 2.5D UNet) and then updating the probabilistic maps within the hybrid data.

Datasets

The authors evaluated their model on two datasets:

  • NIH Pancreas-CT dataset: This dataset was used for extensive quantitative and qualitative evaluation. It contains abdominal CT scans from 82 patients.
  • MSD pancreas dataset (Medical Segmentation Decathlon): This dataset, containing 281 CT scans with labeled pancreas and pancreas tumor, was used to further validate the model's performance on a different dataset.

Key Insights and Ideas

  • Leveraging Probabilistic Maps for Context: Instead of just using raw image data from adjacent slices, using their probabilistic maps provides a more robust and less noisy representation of the pancreas's likely location in neighboring slices. This is a key insight for guiding the segmentation.
  • Bi-directional Recurrence for Temporal Context: The bi-directional recurrent update is crucial. It allows information to flow in both directions (forward and backward through slices), ensuring that the network fully utilizes the contextual information from the entire local neighborhood, not just from one direction. This is analogous to how radiologists might scroll through slices.
  • 2.5D Approach as a Sweet Spot: The PBR-UNet uses a 2.5D UNet. This approach aims to strike a balance between the computational cost of full 3D methods and the contextual limitations of 2D methods, by processing slices in a way that incorporates inter-slice information.
  • Efficiency Gains: The paper highlights that their approach achieves competitive results with less computational cost compared to many state-of-the-art 3D methods, making it more practical for clinical deployment.

Key Results

  • Segmentation Performance: The PBR-UNet achieved a mean Dice Similarity Coefficient (DSC) of 85.35% ± 4.13% on the NIH Pancreas-CT dataset. This is competitive with or better than many existing methods.
  • Localization Performance: The model demonstrated strong localization capabilities with recall, precision, and IOU scores of 82.76%, 83.45%, and 75.03%, respectively.
  • Slice-wise Performance: The method showed significant improvement in segmenting small or challenging slices, reducing the number of failed segmentations by more than half compared to M-FCN.
  • Computational Efficiency: The paper reports inference times of 0.17 s/slice for M-FCN, 0.38 s/slice for C2F, and 0.57 s/slice for PBR-UNet, indicating a reasonable computational overhead for the improved performance.
  • Cross-Dataset Validation: On the MSD dataset, the PBR-UNet achieved a DSC of 85.65%, demonstrating its generalizability.

Limitations and Future Work

  • Initial Estimation Sensitivity: The authors acknowledge that a small number of regions of interest (ROIs) from the initial estimation might not cover all pancreatic regions, potentially limiting performance improvements in the primary segmentation stage. This is a common challenge in stage-wise segmentation methods.
  • Future Work: The primary focus for future work is to address the sensitivity of stage-wise methods to the initial ROI. The goal is to obtain a more robust initial ROI and mitigate the adverse effects of overly large or small initial segmentations on the final outcome.

Summary updated

Abstract

Pancreas segmentation in medical imaging data is of great significance for clinical pancreas diagnostics and treatment. However, the large population variations in the pancreas shape and volume cause enormous segmentation difficulties, even for state-of-the-art algorithms utilizing fully-convolutional neural networks (FCNs). Specifically, pancreas segmentation suffers from the loss of spatial information in 2D methods, and the high computational cost of 3D methods. To alleviate these problems, we propose a probabilistic-map-guided bi-directional recurrent UNet (PBR-UNet) architecture, which fuses intra-slice information and inter-slice probabilistic maps into a local 3D hybrid regularization scheme, which is followed by bi-directional recurrent network optimization. The PBR-UNet method consists of an initial estimation module for efficiently extracting pixel-level probabilistic maps and a primary segmentation module for propagating hybrid information through a 2.5D U-Net architecture. Specifically, local 3D information is inferred by combining an input image with the probabilistic maps of the adjacent slices into multichannel hybrid data, and then hierarchically aggregating the hybrid information of the entire segmentation network. Besides, a bi-directional recurrent optimization mechanism is developed to update the hybrid information in both the forward and the backward directions. This allows the proposed network to make full and optimal use of the local context information. Quantitative and qualitative evaluation was performed on the NIH Pancreas-CT dataset, and our proposed PBR-UNet method achieved better segmentation results with less computational cost compared to other state-of-the-art methods.
BibTeX
@misc{li2019probability,
  author = {Jun Li and Xiaozhu Lin and Hui Che and Hao Li and Xiaohua Qian},
  title = {{Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation}},
  year = {2019},
  eprint = {1903.00923},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1903.00923},
}
Figure