Enhancing Dual Network Based Semi-Supervised Medical Image Segmentation with Uncertainty-Guided Pseudo-Labeling

Yunyao Lu, Yihang Wu, Ahmad Chaddad, Tareef Daqqaq, Reem KatebarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a novel dual-network framework for semi-supervised 3D medical image segmentation that effectively combats the challenge of noisy pseudo-labels. By combining cross-pseudo supervision with adaptive entropy filtering and prototype-guided contrastive learning, the model achieves state-of-the-art segmentation performance using as little as 10% labeled data.

What Problem Does This Paper Address?

Deep learning has revolutionized medical image segmentation, but its success heavily relies on massive amounts of voxel-level annotations. Generating these annotations is exceptionally tedious, time-consuming, and requires expert clinical knowledge. Semi-supervised learning (SSL) offers a solution by training models on a small pool of labeled data alongside a much larger pool of unlabeled data.

A common SSL strategy is pseudo-labeling, where the model generates its own targets for unlabeled images. However, current methods face two major hurdles:

  1. Noisy Pseudo-Labels: Early in training, model predictions are highly unreliable, leading to erroneous pseudo-labels that degrade performance.
  2. Insufficient Feature-Space Supervision: Most methods only enforce consistency on the final output probabilities, ignoring whether the underlying feature representations are actually discriminative. Furthermore, standard fixed confidence thresholds introduce a bias toward "easy" classes (like the background), leaving complex foreground structures poorly segmented.

Methods

The authors tackle these challenges by proposing a dual-stream network architecture consisting of two parallel student networks with different backbones: a ResNet34-based subnet ($f_A$) and a VNet-based subnet ($f_B$).

Figure 2: Pipeline of semi-supervised segmentation framework. Both labeled and unlabeled 3D medical images are simultaneously fed into two 3D encoder-decoder networks to enhance segmentation performance: a ResNet34-based subnet 𝑓𝐴and a VNet-based subnet 𝑓𝐵. The output features from both subnets are used to compute contrastive loss 𝑐by aligning foreground and background prototypes in the feature space. The predicted segmentation maps are supervised using labeled data via a combined cross-entropy and Dice loss (𝑠). A consistency regularization loss (𝑐𝑟) is applied on labeled data to encourage prediction stability, while an uncertainty estimation loss (𝑢𝑛𝑒) is applied on unlabeled data to suppress unreliable pseudo labels. Pseudo labels are generated from the model predictions and further refined using Cross Pseudo Supervision loss (𝑐𝑝𝑠) and Entropy-Filtered Supervision loss (𝑒𝑓𝑠) on unlabeled data.

The framework relies on three core innovations:

  • Cross Consistency Enhancement (CCE): This module combines Cross Pseudo Supervision (CPS)—where Subnet A generates pseudo-labels to supervise Subnet B, and vice versa—with Entropy-Filtered Supervision (EFS). EFS calculates the voxel-wise entropy of predictions and masks out highly uncertain, high-entropy regions to ensure the networks only learn from trustworthy pseudo-labels.
Figure 3: The process of CCE module, which combines Cross Pseudo Supervision (CPS) with Entropy-Filtered Supervision (EFS) to improve the reliability of pseudo-labeling. In CPS, the predictions from one model serve as pseudo-labels for the other, where predictions exceeding a confidence threshold are assigned as 1 and others as 0. These pseudo-labels are then used to supervise the other model. In EFS, entropy maps are computed for each pixel’s predicted probability. Pixels with entropy exceeding a threshold in both models are masked out during training, ensuring that only low-uncertainty regions contribute to learning.
  • Dynamic Uncertainty-Aware Weighting: Instead of treating all pseudo-labels equally, the framework uses Kullback-Leibler (KL) divergence to measure the disagreement between the two subnets. Voxel-wise weights are dynamically assigned so that high-confidence voxels contribute more to the loss, while uncertain ones are down-weighted.
  • Prototype-Guided Contrastive Learning (PGL): To fix the lack of feature-level guidance, the model computes class-specific feature "prototypes" (centroids) from highly confident voxel predictions. It then uses a self-supervised contrastive loss to pull ambiguous, uncertain voxel features closer to their corresponding reliable class prototype while pushing different classes apart.

Datasets

The framework was evaluated on three distinct 3D medical imaging benchmarks:

  1. Left Atrial (LA): 100 3D gadolinium-enhanced MR volumes for left atrium segmentation.
  2. NIH Pancreas: 82 contrast-enhanced 3D CT scans featuring high anatomical variability.
  3. BraTS-2019: 335 brain tumor glioma MRI scans across four modalities (T1, T1Gd, T2, FLAIR).

Key Insights and Ideas

  • Architecture Diversity Prevents Error Compounding: By using two structurally different subnets (ResNet34 vs. V-Net) rather than a traditional identical Teacher-Student setup, the models are less likely to collapse into the same prediction biases. This enables robust, direct inter-network correction without needing auxiliary teacher models.
  • Feature Space Alignment is Crucial: Enforcing consistency solely on final label outputs is a superficial fix. Aligning uncertain voxels with reliable class prototypes in the embedding space ensures that the model learns robust, discriminative representations early on.
  • Adaptive Thresholding Beats Fixed Thresholds: Fixed confidence thresholds inherently favor easy-to-segment background voxels. Utilizing dataset-specific, percentile-based entropy thresholds allows the filtering mechanism to dynamically adapt to the varying difficulty of different anatomical structures.

Key Results

The proposed approach consistently outperformed existing state-of-the-art semi-supervised methods (such as UA-MT, DTC, and MC-Net) across all datasets.

Figure 1: Quantitative (Dice score) and visualization comparisons between our method and recent semi-supervised approaches on the LA, Pancreas, and BraTS-2019 datasets using 10% labeled data.
  • Left Atrial Dataset: With only 10% labeled data (8 scans), the model achieved a 89.95% Dice score, nearly matching the performance of a fully supervised model trained on 100% of the data (90.62%).
  • NIH Pancreas Dataset: On this highly complex dataset, the model achieved a 71.83% Dice score under the 10% labeled setting, outperforming the next-best method by a notable 1.71%.
  • BraTS-2019 Dataset: The model achieved the highest Dice score (84.84%) and the lowest 95th percentile Hausdorff Distance (95HD) under the 10% labeled data split.

Limitations and Future Work

Despite its strong overlap performance (Dice score), the authors note that the model still struggles to achieve optimal results for boundary-specific metrics like Average Symmetric Surface Distance (ASD). Additionally, utilizing a dual-network architecture prioritizes segmentation accuracy over parameter efficiency, making it computationally heavy for edge deployments.

For future work, the authors plan to explore boundary-penalizing loss functions, integrate multi-scale features, and investigate replacing the CNN backbones with more advanced Transformer-based architectures.

Summary updated

Abstract

Despite the remarkable performance of supervised medical image segmentation models, relying on a large amount of labeled data is impractical in real-world situations. Semi-supervised learning approaches aim to alleviate this challenge using unlabeled data through pseudo-label generation. Yet, existing semi-supervised segmentation methods still suffer from noisy pseudo-labels and insufficient supervision within the feature space. To solve these challenges, this paper proposes a novel semi-supervised 3D medical image segmentation framework based on a dual-network architecture. Specifically, we investigate a Cross Consistency Enhancement module using both cross pseudo and entropy-filtered supervision to reduce the noisy pseudo-labels, while we design a dynamic weighting strategy to adjust the contributions of pseudo-labels using an uncertainty-aware mechanism (i.e., Kullback-Leibler divergence). In addition, we use a self-supervised contrastive learning mechanism to align uncertain voxel features with reliable class prototypes by effectively differentiating between trustworthy and uncertain predictions, thus reducing prediction uncertainty. Extensive experiments are conducted on three 3D segmentation datasets, Left Atrial, NIH Pancreas and BraTS-2019. The proposed approach consistently exhibits superior performance across various settings (e.g., 89.95\% Dice score on left Atrial with 10\% labeled data) compared to the state-of-the-art methods. Furthermore, the usefulness of the proposed modules is further validated via ablation experiments.
BibTeX
@misc{lu2025enhancing,
  author = {Yunyao Lu and Yihang Wu and Ahmad Chaddad and Tareef Daqqaq and Reem Kateb},
  title = {{Enhancing Dual Network Based Semi-Supervised Medical Image Segmentation with Uncertainty-Guided Pseudo-Labeling}},
  year = {2025},
  eprint = {2509.13084},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.13084},
}

Loading related content...

Figure