Uncertainty-Guided Pixel Contrastive Learning for Semi-Supervised Medical Image Segmentation

Tao Wang, Jianglin Lu, Zhihui Lai, Jiajun Wen, Heng Kong

Introduction

Deep learning has revolutionized medical image segmentation, yet the performance of these models remains heavily dependent on large-scale, pixel-perfect annotations. In the medical domain, such labels are notoriously difficult to obtain, requiring significant time and expertise from clinicians. Semi-supervised learning (SSL) addresses this "data hunger" by leveraging a small amount of labeled data alongside a vast pool of unlabeled images.

A prominent technique in SSL is contrastive learning, which teaches models to distinguish between similar and dissimilar image regions. However, in semi-supervised settings, the model must rely on its own "pseudo-labels" to identify these regions. If these initial guesses are incorrect—a common occurrence in complex medical scans—the model learns from noisy data, leading to degraded performance. Furthermore, many pixel-level contrastive methods focus exclusively on local textures, often losing the global anatomical context necessary for accurate diagnosis.

Proposed Methodology

We introduce a novel framework that utilizes uncertainty guidance to refine pixel-level contrastive learning, ensuring the model learns from high-confidence regions while maintaining a global understanding of anatomy.

Overview of the proposed framework featuring Equivariant Contrastive Learning, Consistency Learning with heterogeneous decoders, and Uncertainty Guided Pixel Contrastive Learning.

1. Heterogeneous Consistency Learning

Traditional semi-supervised frameworks often use identical network architectures (e.g., the "Mean Teacher" approach). Our method instead employs two structurally different decoders: a CNN-based UNet and a Transformer-based Swin-UNet. Because CNNs prioritize local spatial patterns and Transformers focus on long-range dependencies, their disagreement provides a robust and natural measure of uncertainty. Areas where the two decoders disagree are flagged as "uncertain."

2. Uncertainty-Guided Pixel Contrastive Learning

Using the disagreement between decoders, the system generates an uncertainty map. When sampling pixels for contrastive learning, the framework explicitly masks out these high-uncertainty regions. By filtering the "noise" out of the pseudo-labels, the model focuses its learning capacity on high-confidence pixels, leading to more reliable feature representations.

3. Equivariant Contrastive Loss

To prevent the model from becoming overly focused on local pixel patterns, we incorporate an equivariant contrastive loss. The encoder is tasked with predicting geometric transformations (such as 90-degree rotations) applied to the input. This encourages the network to learn the underlying global geometry and structural relationships of the organs or tissues being imaged.

Experimental Setup and Datasets

The framework was evaluated on two major medical imaging benchmarks:

  • ACDC Dataset: Comprising 200 cardiac MR images. We tested the model's performance with as few as 68 labeled images, representing data from only three patients.
  • ISIC Dataset: Containing 2594 dermoscopy images for skin lesion segmentation. Testing was conducted using limited label sets of 5% and 10%.

Key Results and Insights

Our method consistently outperformed existing state-of-the-art (SOTA) semi-supervised methods, including Mean Teacher (MT) and Cross Pseudo Supervision (CPS).

  • Performance Gains: On the ACDC dataset (68 labels), the model achieved an 80.05% Dice score, significantly higher than the supervised-only baseline (65.32%) and the previous SOTA, CPS (74.76%). On the ISIC dataset (91 labels), it reached 72.67%, surpassing other methods that typically ranged between 66% and 71%.
  • Feature Separation: Visualizing the feature space using t-SNE reveals that our method produces much clearer separation between classes compared to standard pseudo-labeling approaches.
t-SNE visualization comparing feature distributions: Without Contrast, Pseudo Contrast, and Our Method. Our method shows significantly clearer class separation and tighter clusters.

Ablation studies further confirmed that the uncertainty guidance alone provided a ~1% boost in Dice score, proving that "quality over quantity" in pixel sampling is a critical factor for SSL success.

Limitations and Future Work

While the framework sets a new benchmark for semi-supervised segmentation, it involves higher computational costs due to the simultaneous training of two distinct decoders and a contrastive memory queue. Additionally, the uncertainty threshold ($H$) may require specific tuning when applied to different imaging modalities (e.g., CT vs. Ultrasound). Future work will investigate multi-scale feature contrast to further enhance the model's ability to capture both fine-grained details and broad anatomical structures.

Summary updated

Abstract

Recently, contrastive learning has shown great potential in medical image segmentation. Due to the lack of expert annotations, however, it is challenging to apply contrastive learning in semi-supervised scenes. To solve this problem, we propose a novel uncertainty-guided pixel contrastive learning method for semi-supervised medical image segmentation. Specifically, we construct an uncertainty map for each unlabeled image and then remove the uncertainty region in the uncertainty map to reduce the possibility of noise sampling. The uncertainty map is determined by a well-designed consistency learning mechanism, which generates comprehensive predictions for unlabeled data by encouraging consistent network outputs from two different decoders. In addition, we suggest that the effective global representations learned by an image encoder should be equivariant to different geometric transformations. To this end, we construct an equivariant contrastive loss to strengthen global representation learning ability of the encoder. Extensive experiments conducted on popular medical image benchmarks demonstrate that the proposed method achieves better segmentation performance than the state-of-the-art methods.
BibTeX
@inproceedings{wang2022uncertainty,
  author = {Tao Wang and Jianglin Lu and Zhihui Lai and Jiajun Wen and Heng Kong},
  title = {{Uncertainty-Guided Pixel Contrastive Learning for Semi-Supervised Medical Image Segmentation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/201},
}
Figure