Self-paced Sample Selection for Barely-Supervised Medical Image Segmentation

Junming Su, Zhiqiang Shen, Peng Cao, Jinzhu Yang, Osmar R. Zaïane

TL;DR

This paper introduces SPSS, a novel framework for barely-supervised medical image segmentation that tackles the issue of noisy pseudo-labels generated by registration methods. By employing self-paced uncertainty selection in the image space and contrastive learning in the feature space, SPSS effectively improves pseudo-label quality, leading to state-of-the-art performance with minimal annotations.

What Problem Does This Paper Address?

Medical image segmentation, crucial for diagnosis and treatment planning, typically requires vast amounts of pixel-level annotated data. However, obtaining such annotations is extremely labor-intensive and requires expert knowledge, leading to a significant "label scarcity" problem. While semi-supervised learning (SSL) methods have made strides, they struggle with "barely-supervised" scenarios where only a few slices of a 3D volume are annotated. Existing barely-supervised methods often rely on a registration-segmentation paradigm, where a registration module generates 3D pseudo-labels from single-slice annotations. The core challenge here is that these registration-generated pseudo-labels are inherently noisy and inaccurate, which significantly degrades the segmentation performance. This paper aims to address this critical limitation by improving the quality of these noisy pseudo-labels.

Methods

The proposed solution is a Self-Paced Sample Selection framework (SPSS), which comprises two key components trained collaboratively in a self-paced learning manner:

  1. Self-Paced Uncertainty Sample Selection (SU): This component focuses on improving pseudo-label quality in the image space at the voxel level. Instead of using fixed thresholds for selecting reliable pseudo-labels, SU gradually selects high-quality pseudo-labels from easy to hard. It achieves this by calculating an uncertainty map for each voxel and using a self-paced mask to select the most confident voxels. The selection process is guided by a warm-up mechanism that adapts to the model's learning status, ensuring that as the model becomes more reliable, it incorporates more challenging but still high-quality samples.
  2. Self-Paced Bidirectional Feature Contrastive Learning (SC): This component aims to implicitly improve pseudo-label quality by enhancing the separability of class semantics in the feature space. It uses the self-paced mask generated by SU to select appropriate positive and negative samples for contrastive learning. By pulling together similar features (positive samples) and pushing apart dissimilar ones (negative samples), SC encourages the model to learn more discriminative feature representations. This, in turn, helps to refine the pseudo-labels by making them more robust to registration noise.

Both SU and SC are trained together using a self-paced learning strategy, allowing the framework to progressively learn from increasingly reliable pseudo-labels.

Datasets

The paper evaluates SPSS on two public medical image segmentation datasets:

  • 2018 Left Atrial Segmentation Challenge (LA) dataset: Used for left atrial segmentation.
  • 2019 Kidney Segmentation Challenge (KiTS) dataset: Used for kidney segmentation.

The authors randomly split these datasets into training and testing sets, using a small fraction of labeled data (e.g., 16 slices for LA, 38 slices for KiTS) for the barely-supervised setting.

Key Insights and Ideas

  • Pseudo-label quality is paramount: The paper strongly emphasizes that the noisy pseudo-labels generated by registration are the bottleneck in existing barely-supervised segmentation methods. Improving their quality is key to achieving better performance.
  • Dual-space improvement: SPSS's innovation lies in simultaneously improving pseudo-label quality in both the image (voxel-level) and feature (semantic representation) spaces. This holistic approach leads to more robust segmentation.
  • Self-paced learning for adaptability: The self-paced learning mechanism is crucial for adapting the sample selection process to the model's evolving confidence and learning stage, preventing premature reliance on potentially noisy samples.
  • Contrastive learning for feature refinement: Using contrastive learning to enhance feature separability provides an indirect but effective way to mitigate registration noise and improve the semantic understanding of the image.

Key Results

  • State-of-the-art performance: SPSS achieves superior performance compared to existing barely-supervised and even some semi-supervised methods on both the LA and KiTS datasets.
  • Significant DSC improvements: On the LA dataset, SPSS achieved a 2.18% improvement in Dice Similarity Coefficient (DSC) over the state-of-the-art PLN method. On the KiTS dataset, it obtained a DSC gain of 1.22% over PLN.
  • Efficiency with minimal labels: Remarkably, SPSS, using only 16 labeled slices for LA, achieved comparable performance to a semi-supervised method that utilized 1280 labeled slices.
  • Qualitative improvements: Qualitative results show that SPSS produces more accurate and smoother segmentation boundaries, particularly in challenging regions.

Limitations and Future Work

  • Reliance on registration: While SPSS mitigates the impact of noisy pseudo-labels, it still relies on the initial registration-segmentation paradigm. The inherent limitations of the registration module might still pose a ceiling on performance.
  • Hyperparameter tuning: The paper mentions that hyperparameters like the temperature coefficient, warm-up function, and age parameters require investigation, suggesting that optimal settings might be dataset-dependent.

Future work could explore more robust registration techniques or alternative paradigms that are less sensitive to registration noise. Further investigation into the optimal self-paced learning schedules and contrastive learning strategies could also yield additional improvements.

Summary updated

Abstract

The existing barely-supervised medical image segmentation (BSS) methods, adopting a registration-segmentation paradigm, aim to learn from data with very few annotations to mitigate the extreme label scarcity problem. However, this paradigm poses a challenge: pseudo-labels generated by image registration come with significant noise. To address this issue, we propose a self-paced sample selection framework (SPSS) for BSS. Specifically, SPSS comprises two main components: 1) self-paced uncertainty sample selection (SU) for explicitly improving the quality of pseudo labels in the image space, and 2) self-paced bidirectional feature contrastive learning (SC) for implicitly improving the quality of pseudo labels through enhancing the separability between class semantics in the feature space. Both SU and SC are trained collaboratively in a self-paced learning manner, ensuring that SPSS can learn from high-quality pseudo labels for BSS. Extensive experiments on two public medical image segmentation datasets demonstrate the effectiveness and superiority of SPSS over the state-of-the-art. Our code is release at https://github.com/SuuuJM/SPSS.
BibTeX
@inproceedings{su2024self,
  author = {Junming Su and Zhiqiang Shen and Peng Cao and Jinzhu Yang and Osmar R. Zaïane},
  title = {{Self-paced Sample Selection for Barely-Supervised Medical Image Segmentation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2024},
  url = {https://openreview.net/forum?id=1SZkDUw5T3},
}
Figure