Semi-supervised Segmentation via Uncertainty Rectified Pyramid Consistency and Its Application to Gross Target Volume of Nasopharyngeal Carcinoma

Xiangde Luo, Wenjun Liao, Jieneng Chen, Tao Song, Yinan Chen, Guotai Wang, Shaoting Zhang

TL;DR

This paper introduces the Uncertainty Rectified Pyramid Consistency (URPC) framework for semi-supervised segmentation of Nasopharyngeal Carcinoma (NPC) tumors. URPC leverages multi-scale predictions from a single network and an uncertainty-rectifying module to effectively utilize unlabeled data, achieving performance close to fully supervised methods with significantly less labeled data.

What Problem Does This Paper Address?

Accurate segmentation of Gross Target Volume (GTV) in Nasopharyngeal Carcinoma (NPC) is crucial for effective radiotherapy planning. While deep learning models like Convolutional Neural Networks (CNNs) excel at this task, they typically require a large amount of meticulously annotated data. Acquiring such datasets is expensive and time-consuming, especially for medical imaging where expert annotation is essential. This paper addresses the challenge of achieving high-performance GTV segmentation with limited labeled data by developing a semi-supervised approach that effectively utilizes readily available unlabeled data.

Methods

The core of the proposed method is the Uncertainty Rectified Pyramid Consistency (URPC) framework. It builds upon a standard 3D UNet backbone, modified to produce pyramid predictions (PPNet) at multiple scales within its decoder.

Here's a breakdown of the key components:

  • Pyramid Prediction Network (PPNet): The decoder of the backbone network is augmented to output predictions at various resolutions (scales). This allows the model to capture both low-frequency (coarse) and high-frequency (fine) details of the segmentation.
  • Supervised Loss: For labeled data, a standard loss function combining Dice and cross-entropy is applied across all prediction scales.
  • Pyramid Consistency Loss: For unlabeled data, the framework encourages consistency among the multi-scale predictions. The idea is that predictions for the same input at different scales should be similar.
  • Uncertainty Rectification: A novel module is introduced to address the issue that directly enforcing consistency across different scales can be problematic due to varying resolutions and potential noise in predictions. This module estimates the uncertainty of each prediction at each scale by measuring the discrepancy (KL-divergence) between the scale-specific prediction and the average prediction across all scales. This uncertainty is then used to rectify the consistency loss, down-weighting unreliable predictions (high uncertainty) and emphasizing reliable ones (low uncertainty). This is achieved through a voxel- and scale-wise weight, $w = e^{-D^s}$, where $D^s$ is the uncertainty at scale $s$.
  • Uncertainty Minimization: An additional term is included to further reduce prediction variance during training.

The overall loss function combines the supervised loss for labeled data and the uncertainty rectified pyramid consistency loss for unlabeled data, with a time-dependent weighting factor to balance their contributions during training.

Datasets

The study utilizes a dataset of 258 T1-weighted MRI images from patients with Nasopharyngeal Carcinoma (NPC) collected from West China Hospital. The dataset was split into:

  • Training set: 180 cases
  • Validation set: 20 cases
  • Testing set: 58 cases

For the semi-supervised experiments, only a small fraction of the training data was labeled:

  • 10% labeled data (18 images) with the remaining 162 as unlabeled.
  • Experiments were also conducted with 20% and 50% labeled data.

Key Insights and Ideas

  • Multi-scale predictions capture complementary information: By generating predictions at different resolutions, the model can leverage both coarse and fine details for segmentation, which is beneficial for complex structures like tumors.
  • Uncertainty estimation from prediction discrepancy is efficient: Instead of computationally expensive methods like Monte Carlo dropout, the paper proposes estimating uncertainty by simply comparing predictions from different scales within a single forward pass. This significantly reduces computational overhead.
  • Uncertainty-guided consistency is crucial for stability: Directly enforcing consistency across scales can be unstable due to noise and resolution differences. Rectifying the consistency loss based on estimated uncertainty allows the model to learn more robustly by focusing on reliable regions.
  • Semi-supervised learning can bridge the gap to fully supervised performance: The results demonstrate that with a significant portion of unlabeled data, semi-supervised methods can achieve performance remarkably close to fully supervised training, drastically reducing the need for extensive manual annotation.

Key Results

  • Improved performance with limited labels: With only 10% or 20% labeled data, URPC significantly improved segmentation performance (measured by Dice Similarity Coefficient - DSC and Average Surface Distance - ASD) compared to supervised learning with the same limited labels.
  • Outperforms state-of-the-art: URPC outperformed five other state-of-the-art semi-supervised segmentation methods when using 10% labeled data.
  • Near fully supervised performance: When 50% of the images were labeled, URPC achieved a mean DSC of 82.74%, which is very close to the fully supervised performance of 83.51%.
  • Ablation studies confirm effectiveness: An ablation study showed that both the uncertainty rectification (UR) and uncertainty minimization (UM) terms, along with an increasing number of pyramid scales (up to 4), contributed to improved performance.
  • Efficient inference: The method requires only a single forward pass per image for both prediction and uncertainty estimation, making it computationally efficient.

Limitations and Future Work

  • Optimal number of scales: While increasing scales generally improved performance, using 5 scales resulted in slightly lower performance than 4, suggesting that too many scales might not always be beneficial, possibly due to loss of detail at very low resolutions. The optimal number of scales might be dataset-dependent.
  • Generalizability: The paper focused on NPC GTV segmentation. The authors suggest that the framework's effectiveness indicates its potential for segmenting other lesions and organs without relying on task-specific knowledge.
  • Future work: The authors plan to evaluate the framework on other segmentation tasks and explore its application beyond medical imaging. They also mention that while their method is efficient, other approaches like multi-head networks can also produce multiple predictions but may increase model capacity and GPU memory requirements.

Summary updated

Abstract

Gross Target Volume (GTV) segmentation plays an irreplaceable role in radiotherapy planning for Nasopharyngeal Carcinoma (NPC). Despite that Convolutional Neural Networks (CNN) have achieved good performance for this task, they rely on a large set of labeled images for training, which is expensive and time-consuming to acquire. In this paper, we propose a novel framework with Uncertainty Rectified Pyramid Consistency (URPC) regularization for semi-supervised NPC GTV segmentation. Concretely, we extend a backbone segmentation network to produce pyramid predictions at different scales. The pyramid predictions network (PPNet) is supervised by the ground truth of labeled images and a multi-scale consistency loss for unlabeled images, motivated by the fact that prediction at different scales for the same input should be similar and consistent. However, due to the different resolution of these predictions, encouraging them to be consistent at each pixel directly has low robustness and may lose some fine details. To address this problem, we further design a novel uncertainty rectifying module to enable the framework to gradually learn from meaningful and reliable consensual regions at different scales. Experimental results on a dataset with 258 NPC MR images showed that with only 10% or 20% images labeled, our method largely improved the segmentation performance by leveraging the unlabeled images, and it also outperformed five state-of-the-art semi-supervised segmentation methods. Moreover, when only 50% images labeled, URPC achieved an average Dice score of 82.74% that was close to fully supervised learning.
BibTeX
@inproceedings{luo2021semi,
  author = {Xiangde Luo and Wenjun Liao and Jieneng Chen and Tao Song and Yinan Chen and Guotai Wang and Shaoting Zhang},
  title = {{Semi-supervised Segmentation via Uncertainty Rectified Pyramid Consistency and Its Application to Gross Target Volume of Nasopharyngeal Carcinoma}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  url = {https://openreview.net/forum?id=b1PhH7BEQpp},
}
Figure