A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation

Hongfeng You, Shengwei Tian, Long Yu, Xiang Ma, Yan Xing, Xin Ning

Abstract

To better retain the deep features of an image and solve the sparsity problem of the end-to-end segmentation model, we propose a new deep convolutional network model for medical image pixel segmentation, called MC-Net. The core of this network model consists of four parts: an encoder network, a multiple max-pooling integration module, a cross multiscale deconvolution decoder network, and a pixel-level classification layer. By employing multiscale convolution and a novel integration strategy, MC-Net achieves translation invariance and solves the sparsity problem after max-pooling. Experimental results on Kaggle, CHAOS, and BraTS datasets demonstrate that MC-Net outperforms several state-of-the-art segmentation networks.

Introduction

Medical image segmentation is a high-stakes task requiring extreme precision for applications such as identifying tumor boundaries or isolating individual nuclei. Traditional deep learning models, including Fully Convolutional Networks (FCNs) and U-Net, often struggle with the "sparsity problem." This occurs during aggressive downsampling (max-pooling), where fine-grained spatial information is discarded, resulting in sparse feature maps in deeper layers. This sparsity makes it difficult for the decoder to accurately reconstruct sharp, accurate boundaries. MC-Net addresses this by rethinking the encoding and decoding processes through a multiscale fusion approach.

Methods

The MC-Net architecture is divided into three primary stages as shown in Figure 1:

  1. Multiscale Encoder: Instead of standard single-channel convolutions, each submodule in the encoder utilizes three different kernel sizes (2x2, 3x3, and 4x4) in parallel. This allows the network to capture features at multiple granularities simultaneously.
  2. Multiple Max-pooling Integration (MMI) Module: This module integrates outputs from all encoder submodules. It uses 1x1 convolutions to maintain parameter efficiency and applies various pooling sizes (ranging from 1 to 32) to capture global context and ensure translation invariance.
  3. Cross Multiscale Deconvolution (CMD) Decoder: The decoder employs a "cross-fusion" strategy. It fuses multiscale features from the encoder with upsampled features in the decoder, using encoder features as "prior knowledge" for the reconstruction phase.
The overall architecture of MC-Net, featuring the multiscale encoder, integration module, and cross-fused decoder.

Experimental Setup and Datasets

The model was evaluated on three distinct medical imaging challenges to prove its versatility:

  • Kaggle 2018 Data Science Bowl: A binary segmentation task focused on identifying cell nuclei.
  • CHAOS Dataset: A multiclass task involving abdominal organs (Liver, Kidneys, and Spleen) from MRI scans.
  • BraTS 2018 Dataset: A complex multiclass task for brain tumor segmentation across four MRI modalities (FLAIR, T1, T1ce, T2).

Key Results

MC-Net consistently outperformed established models across all benchmarks. On the Kaggle dataset, it achieved a Dice score of 98.81%, surpassing U-Net (98.2%) and MS-Dual (98.66%). In the CHAOS challenge, it showed significant improvements in Spleen segmentation (92.05% Dice) compared to MS-Dual (78.61%).

Performance comparison of MC-Net against state-of-the-art models on the Kaggle 2018 Data Science Bowl dataset.

Qualitative analysis revealed that MC-Net produces smoother boundaries and superior denoising compared to FCNs or SegNet, which often yield blurry or noisy edges. Despite its multiscale complexity, the model remains efficient with only 6.8 million parameters, significantly fewer than SegNet (10M) or PSPNet (10M).

Conclusion

MC-Net introduces a robust framework for medical image segmentation by effectively addressing the sparsity problem through multiscale integration and cross-fusion. While internal configurations like the order of Batch Normalization and ReLU may need adjustment based on the specific imaging modality, the architecture provides a powerful foundation for complex multi-modal medical imaging tasks.

Summary updated

Abstract

To better retain the deep features of an image and solve the sparsity problem of the end-to-end segmentation model, we propose a new deep convolutional network model for medical image pixel segmentation, called MC-Net. The core of this network model consists of four parts, namely, an encoder network, a multiple max-pooling integration module, a cross multiscale deconvolution decoder network and a pixel-level classification layer. In the network structure of the encoder, we use multiscale convolution instead of the traditional single-channel convolution. The multiple max-pooling integration module first integrates the output features of each submodule of the encoder network and reduces the number of parameters by convolution using a kernel size of 1. At the same time, each max-pooling layer (the pooling size of each layer is different) is spliced after each convolution to achieve the translation invariance of the feature maps of each submodule. We use the output feature maps from the multiple max-pooling integration module as the input of the decoder network; the multiscale convolution of each submodule in the decoder network is cross-fused with the feature maps generated by the corresponding multiscale convolution in the encoder network. Using the above feature map processing methods solves the sparsity problem after the max-pooling layer-generating matrix and enhances the robustness of the classification. We compare our proposed model with the well-known Fully Convolutional Networks for Semantic Segmentation (FCNs), DecovNet, PSPNet, U-net, SgeNet and other state-of-the-art segmentation networks such as HyperDenseNet, MS-Dual, Espnetv2, Denseaspp using one binary Kaggle 2018 data science bowl dataset and two multiclass dataset and obtain encouraging experimental results.
BibTeX
@misc{you2020new,
  author = {Hongfeng You and Shengwei Tian and Long Yu and Xiang Ma and Yan Xing and Xin Ning},
  title = {{A New Multiple Max-pooling Integration Module and Cross Multiscale Deconvolution Network Based on Image Semantic Segmentation}},
  year = {2020},
  eprint = {2003.11213},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2003.11213},
}
Figure