Nested Dilation Network (NDN) for Multi-Task Medical Image Segmentation

Liansheng Wang, Rongzhen Chen, Shuxin Wang, Nianyin Zeng, Xiaoyang Huang, Changhua LiuIEEE Access 2019 · OpenReview

Abstract

The deep convolutional network has shown excellent performance in medical image analysis. However, almost all network variants are presented for one specific task, e.g., segment pancreas on computerized tomography (CT). In this paper, we propose a nested dilation network (NDN) which is applied to multiple segmentation tasks even for different modalities, including CT, magnetic resonance imaging (MRI), and endoscopic images. We design residual blocks nested with dilations (RnD Blocks) that catch larger receptive field in the first few layers to boost shallow semantic information. Besides, we apply the modified focal loss to help the network to provide more accurate segmentation results. We evaluate our method on five subtasks from medical segmentation decathlon challenge and GIANA2018, and the results show that our method achieves a better performance than the latest methods in each task. A lot of research works have been done recently to strengthen the learning power of the convolutional neural network (CNN) to get better performance.

Introduction

In the clinical environment, medical image segmentation is a vital step for diagnosis and treatment planning. However, developing a robust model that generalizes across different anatomical structures and imaging modalities remains a significant challenge. Most current state-of-the-art models are "one-trick ponies," optimized for a single task like identifying a brain tumor on an MRI but failing when applied to CT scans or endoscopic videos. The Nested Dilation Network (NDN) is designed as a versatile "Swiss Army knife" for medical imaging, capable of high performance across diverse clinical benchmarks without extensive task-specific re-engineering.

The Challenge of Generalization

Creating a multi-task segmentation model requires addressing three core issues:

  1. Scale Variance: Targets range from massive organs (e.g., the liver) to tiny, localized lesions (e.g., polyps).
  2. Modality Differences: Imaging techniques like CT and MRI represent anatomy using fundamentally different signal characteristics.
  3. Class Imbalance: In many medical images, the target of interest occupies less than 1% of the total pixels, often leading standard models to ignore the foreground in favor of the background.

The NDN Architecture

The NDN architecture utilizes a U-Net backbone enhanced by three critical modifications designed to improve context capture and handle imbalance.

RnD Blocks (Residual blocks nested with Dilations)

Standard convolutional neural networks (CNNs) often suffer from a limited receptive field in their shallow layers, which can result in the propagation of low-level noise through skip connections. The NDN introduces RnD Blocks, which stack three dilated convolutions with dilation rates of 1, 2, and 4 within a residual framework. This configuration allows the network to achieve a 15x15 receptive field in its earliest layers, capturing broad spatial context while maintaining fine-grained details and avoiding the "gridding effect" artifacts often seen in standard dilated convolutions.

Structure of the RnD block showing nested dilated convolutions and the resulting expansion of the receptive field.

Squeeze-and-Excitation (SE) Blocks

To handle modality differences, SE blocks are embedded throughout the network. These act as a channel-wise attention mechanism, allowing the model to recalibrate feature importance dynamically. This helps the network focus on the most relevant textures and signals for a specific task, whether it is an MRI, CT, or endoscopic image.

Modified Focal Loss

To solve the "needle in a haystack" problem of class imbalance, the authors implemented a modified focal loss. Unlike standard loss functions that may become biased toward the dominant background class, this version applies a high penalty to error pixels regardless of their class, forcing the model to provide more accurate results for small or fuzzy targets.

Results and Key Insights

The NDN was evaluated on five diverse subtasks, including brain tumor (MRI), hippocampus (MRI), liver tumor (CT), pancreas (CT), and colon polyps (endoscopy). The results demonstrated superior performance across the board:

  • Liver Tumor: Achieved a Dice score of 0.6795, a substantial improvement over previous DCNN (0.4955) and CFCN (0.4342) models.
  • Pancreas: Reached a Dice score of 0.8476, outperforming the 3D U-Net.
  • Colon Polyps: Proved the architecture's versatility by achieving a Dice score of 0.8934 on 2D endoscopic tasks.

A key insight from this research is that capturing context in the shallow layers is vital. By doing so, the NDN ensures that the information passed through the network's skip connections is semantically rich, rather than just low-level noise.

Conclusion

The Nested Dilation Network offers a robust solution for multi-task medical image segmentation. By combining expanded receptive fields in early layers with adaptive attention and a specialized loss function, the NDN achieves state-of-the-art results across multiple modalities. Future work may explore integrating more advanced attention mechanisms or 2.5D approaches to further balance computational efficiency with 3D spatial context.

Summary updated

Abstract

The deep convolutional network has shown excellent performance in medical image analysis. However, almost all network variants are presented for one specific task, e.g., segment pancreas on computerized tomography (CT). In this paper, we propose a nested dilation network (NDN) which is applied to multiple segmentation tasks even for different modalities, including CT, magnetic resonance imaging (MRI), and endoscopic images. We design residual blocks nested with dilations (RnD Blocks) that catch larger receptive field in the first few layers to boost shallow semantic information. Besides, we apply the modified focal loss to help the network to provide more accurate segmentation results. We evaluate our method on five subtasks from medical segmentation decathlon challenge and GIANA2018, and the results show that our method achieves a better performance than the latest methods in each task. A lot of research works have been done recently to strengthen the learning power of the convolutional neural network (CNN) to get better performance.
BibTeX
@inproceedings{wang2019nested,
  author = {Liansheng Wang and Rongzhen Chen and Shuxin Wang and Nianyin Zeng and Xiaoyang Huang and Changhua Liu},
  title = {{Nested Dilation Network (NDN) for Multi-Task Medical Image Segmentation}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=NophkI80X1XC},
}
Figure