A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images

Zijie Chen, Cheng Li, Junjun He, Jin Ye, Diping Song, Shanshan Wang, Lixu Gu, Yu Qiao

Abstract

Radiation therapy (RT) planning for head and neck (HaN) cancers requires precise segmentation of various organs-at-risk (OARs). Manual segmentation is labor-intensive and prone to variability across hundreds of CT slices. While 3D convolutional neural networks (CNNs) are commonly used for this task, they often struggle with the anisotropic resolution of CT scans and the extreme class imbalance between large and small organs. We propose OrganNet2.5D, a hybrid CNN that fuses 2D and 3D convolutions to handle varying spatial resolutions and maintain high-fidelity edge features. By limiting downsampling and employing hybrid dilated convolutions (HDC), OrganNet2.5D preserves spatial detail for small organs while maintaining a sufficient receptive field. Experiments on the MICCAI 2015 challenge dataset demonstrate that our model achieves state-of-the-art performance, particularly for challenging small structures.

Introduction

In clinical radiation therapy, accurately mapping OARs—such as the brain stem, eyes, and optic nerves—is critical to maximizing tumor dosage while sparing healthy tissue. However, 3D CT images present two significant challenges for automated segmentation:

  1. Anisotropic Resolution: Depth resolution (Z-axis) is typically much lower than in-plane resolution (XY plane), leading to distorted feature extraction in standard 3D CNNs.
  2. Class Imbalance: Massive organs like the mandible coexist with tiny structures like the lens of the eye. Standard deep architectures often downsample images to a degree that these small organs lose their defining features.

Methodology

OrganNet2.5D Architecture

The proposed OrganNet2.5D architecture is designed to respect the high-resolution in-plane slices of CT data before transitioning into 3D feature space. The network begins and ends with 2D convolution blocks to extract sharp edge features from individual slices.

The OrganNet2.5D architecture, featuring hybrid 2D/3D convolutions and a reduced downsampling scheme.

Unlike the classic U-Net, which typically employs four downsampling operations, OrganNet2.5D uses only two. This preservation of spatial resolution is crucial for detecting small organs. To compensate for the reduced receptive field that usually accompanies less downsampling, the model incorporates Hybrid Dilated Convolutions (HDC). HDC allows the network to capture global context without the "gridding effect" artifacts common in standard dilated convolutions. Additionally, 3D blocks utilize Residual Squeeze-and-Excitation (ResSE) units to prioritize important feature channels and suppress noise.

Experimental Results

Datasets and Evaluation

The model was validated using a large internal dataset of 307 samples and the public MICCAI 2015 Challenge Dataset (48 samples). Evaluation focused on the Dice Similarity Coefficient (DSC), a measure of overlap between the predicted and ground-truth segmentations.

Performance Comparison

OrganNet2.5D demonstrated exceptional performance on small, difficult-to-segment organs. On the MICCAI 2015 benchmark, it achieved the highest accuracy for the optic chiasm with a Dice score of 66.3%, outperforming significantly more complex models.

Comparison of OrganNet2.5D with state-of-the-art methods on the MICCAI 2015 dataset.

While models utilizing automated Neural Architecture Search (NAS), such as SCAA, achieved a slightly higher mean DSC (82.6% vs. 81.3%), OrganNet2.5D remains highly competitive while being significantly easier to implement and train. Ablation studies further confirmed that the hybrid "2.5D" approach consistently outperformed pure 3D implementations, validating the strategy of using 2D layers to calibrate for anisotropic resolution.

Conclusion

OrganNet2.5D addresses the core challenges of 3D HaN CT segmentation through a strategic blend of 2D and 3D convolutions and a resolution-preserving downsampling strategy. Its superior performance on small organs like the optic chiasm makes it a valuable tool for improving the precision of radiation therapy planning. Future work will explore integrating more advanced attention mechanisms to further bridge the gap with compute-intensive NAS-based models.

Summary updated

Abstract

Radiation therapy (RT) is widely employed in the clinic for the treatment of head and neck (HaN) cancers. An essential step of RT planning is the accurate segmentation of various organs-at-risks (OARs) in HaN CT images. Nevertheless, segmenting OARs manually is time-consuming, tedious, and error-prone considering that typical HaN CT images contain tens to hundreds of slices. Automated segmentation algorithms are urgently required. Recently, convolutional neural networks (CNNs) have been extensively investigated on this task. Particularly, 3D CNNs are frequently adopted to process 3D HaN CT images. There are two issues with na\"ive 3D CNNs. First, the depth resolution of 3D CT images is usually several times lower than the in-plane resolution. Direct employment of 3D CNNs without distinguishing this difference can lead to the extraction of distorted image features and influence the final segmentation performance. Second, a severe class imbalance problem exists, and large organs can be orders of times larger than small organs. It is difficult to simultaneously achieve accurate segmentation for all the organs. To address these issues, we propose a novel hybrid CNN that fuses 2D and 3D convolutions to combat the different spatial resolutions and extract effective edge and semantic features from 3D HaN CT images. To accommodate large and small organs, our final model, named OrganNet2.5D, consists of only two instead of the classic four downsampling operations, and hybrid dilated convolutions are introduced to maintain the respective field. Experiments on the MICCAI 2015 challenge dataset demonstrate that OrganNet2.5D achieves promising performance compared to state-of-the-art methods.
BibTeX
@inproceedings{chen2021novel,
  author = {Zijie Chen and Cheng Li and Junjun He and Jin Ye and Diping Song and Shanshan Wang and Lixu Gu and Yu Qiao},
  title = {{A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87193-2\_54},
}
Figure