RP-Net: A 3D Convolutional Neural Network for Brain Segmentation From Magnetic Resonance Imaging

Liansheng Wang, Cong Xie, Nianyin ZengIEEE Access 2019 · OpenReview

Abstract

Quantitative analysis of brain volume is quite significant for the diagnosis of brain diseases. Accurate segmentation of essential brain tissues from 3D medical images is fundamental to quantitative brain analysis. Since manual segmentation is extremely tedious and time-consuming, there is a growing demand for automated segmentation. In this paper, we propose a 3D convolutional neural network including recursive residual blocks and a pyramid pooling module (RP-Net) for segmenting brain from 3D magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). RP-Net is an U-Net like the network that consists of a downsampling path and an upsampling path. Each path consists of four stages with a recursive residual block. All layers in RP-Net are implemented in a 3D manner. The pyramid pooling module is applied before a voxel-wise classification layer for obtaining both local and global context information. The RP-Net has been evaluated on WM, GM, and CSF segmentation with CANDI, IBSR18, and IBSR20 dataset. The experiments show that the RP-Net achieved mean dice similarity coefficients of 90.7% on CANDI, 90.49% on IBSR18 and 84.96% on IBSR20. The results demonstrate that our proposed method has achieved a significant improvement in segmentation accuracy compared to other reported approaches.

Introduction

Quantitative analysis of brain volume is a cornerstone for diagnosing neurodegenerative diseases like Alzheimer’s and dementia. However, the "gold standard" for this analysis—manual segmentation by radiologists—is an incredibly tedious, slice-by-slice process prone to human error. Automating this task is difficult because MRI scans often suffer from fuzzy tissue edges, noise, and significant anatomical variations between individuals. While deep learning has made strides in this area, standard models often struggle to balance the need for local precision with global context.

The RP-Net Architecture

The authors introduce RP-Net, a 3D architecture based on the popular U-Net framework, but with critical innovations designed for volumetric medical data. Unlike standard 2D approaches, all layers in RP-Net are implemented in a 3D manner to preserve spatial relationships across slices.

The architecture of RP-Net, showing the downsampling and upsampling paths with residual connections and element-wise additions.

Recursive Residual Blocks

Unlike standard residual blocks that simply add an input to an output, RP-Net utilizes recursive residual blocks. This allows the network to learn highly complex features by passing data through the same weights multiple times. This approach effectively deepens the network, allowing for more sophisticated feature extraction without a massive explosion in the number of parameters.

3D Pyramid Pooling Module (PPM)

Situated at the bottleneck of the network, the PPM performs 3D pooling at four different scales. This allows the model to "see" the brain at multiple resolutions simultaneously, fusing local details with global spatial information. This is particularly effective for segmenting Cerebrospinal Fluid (CSF), which often appears in thin, complex pockets easily missed by models looking only at local patches.

The 3D Pyramid Pooling Module (PPM) used to capture multi-scale context information through parallel pooling and convolution operations.

Deep Supervision and Inference Strategy

To improve training stability, the authors added auxiliary classifiers at each stage of the upsampling path. These "checkpoints" help gradients flow better and ensure hidden layers learn meaningful features. During inference, a two-stage strategy is used: the model first performs a coarse pass to locate the brain, then crops the image to focus only on the region of interest, reducing noise from non-brain pixels like the skull or neck.

Experimental Results

The model was rigorously evaluated on three benchmarks: CANDI (pediatric subjects), IBSR18 (high-resolution scans), and IBSR20 (low-resolution scans with more artifacts).

RP-Net consistently outperformed existing state-of-the-art methods like 3D U-Net and VoxResNet. On the IBSR18 dataset, RP-Net achieved an average Dice Similarity Coefficient (DSC) improvement of 4.54% over VoxResNet and 1.85% over the standard 3D U-Net.

Dataset Mean Dice Similarity Coefficient (DSC)
CANDI 90.7%
IBSR18 90.49%
IBSR20 84.96%

Conclusion

RP-Net demonstrates that combining recursive structures with multi-scale pooling significantly improves 3D brain tissue segmentation. While the model is computationally intensive—requiring high-end hardware for training due to the memory demands of 3D convolutions—it sets a new standard for accuracy across both high and low-resolution MRI datasets. Future work will likely focus on clinical integration and further improving robustness to image artifacts.

Summary updated

Abstract

Quantitative analysis of brain volume is quite significant for the diagnosis of brain diseases. Accurate segmentation of essential brain tissues from 3D medical images is fundamental to quantitative brain analysis. Since manual segmentation is extremely tedious and time-consuming, there is a growing demand for automated segmentation. In this paper, we propose a 3D convolutional neural network including recursive residual blocks and a pyramid pooling module (RP-Net) for segmenting brain from 3D magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). RP-Net is an U-Net like the network that consists of a downsampling path and an upsampling path. Each path consists of four stages with a recursive residual block. All layers in RP-Net are implemented in a 3D manner. The pyramid pooling module is applied before a voxel-wise classification layer for obtaining both local and global context information. The RP-Net has been evaluated on WM, GM, and CSF segmentation with CANDI, IBSR18, and IBSR20 dataset. The experiments show that the RP-Net achieved mean dice similarity coefficients of 90.7% on CANDI, 90.49% on IBSR18 and 84.96% on IBSR20. The results demonstrate that our proposed method has achieved a significant improvement in segmentation accuracy compared to other reported approaches.
BibTeX
@inproceedings{wang2019rp,
  author = {Liansheng Wang and Cong Xie and Nianyin Zeng},
  title = {{RP-Net: A 3D Convolutional Neural Network for Brain Segmentation From Magnetic Resonance Imaging}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=ySS27Gy9CxTb},
}
Figure