Learning robust medical image segmentation from multi-source annotations

Yifeng Wang, Luyang Luo, Mingxiang Wu, Qiong Wang, Hao ChenMedical Image Analysis 2025 · DOI

TL;DR

This paper introduces UMA-Net, a novel deep learning framework for medical image segmentation that effectively handles noisy and varied annotations from multiple sources. By employing pixel-level uncertainty estimation and image-level quality assessment, UMA-Net learns from both high-quality and low-quality samples, achieving state-of-the-art performance across diverse medical imaging tasks.

What Problem Does This Paper Address?

Medical image segmentation is crucial for computer-aided diagnosis, but training accurate segmentation models typically requires large datasets with high-quality, single-source annotations. However, obtaining such annotations is challenging and expensive. When multiple annotators or automated methods contribute, their annotations often exhibit variations due to individual preferences, expertise levels, or algorithmic biases, leading to noisy and inconsistent labels. Existing methods that fuse these multi-source annotations often overlook valuable information within the variations and struggle with low-quality samples, potentially degrading model performance. This paper aims to address the challenge of learning robust segmentation models from these inherently noisy and diverse multi-source annotations, without discarding potentially useful information from lower-quality sources.

Methods

The core of UMA-Net lies in its two-pronged uncertainty-guided approach:

  • Pixel-Level Guidance (Annotation Uncertainty Estimation Module - AUEM): AUEM estimates pixel-wise uncertainty maps for each annotation. By analyzing the agreement and disagreement among multiple annotations for the same image, it identifies pixels that are reliably segmented versus those that are ambiguous or noisy. This uncertainty information is then used to weight the segmentation loss (both cross-entropy and Dice loss), effectively down-weighting unreliable pixels and focusing the learning process on more trustworthy ones. A consistency loss is also introduced to encourage agreement among the uncertainty estimators.
  • Image-Level Guidance (Quality Assessment Module - QAM): QAM assesses the overall quality of each image sample based on the derived calibration maps (coarse estimations of the latent ground truth). It categorizes samples into "high-quality" and "low-quality" groups using confidence and agreement scores. High-quality samples are directly used to train the primary segmentation predictor. Crucially, to avoid discarding valuable information from low-quality samples, an auxiliary predictor is introduced. This auxiliary predictor learns from the low-quality samples, allowing the backbone network to extract representation knowledge from them without directly impacting the primary predictor's accuracy. During inference, only the primary predictor and the backbone are used, making the additional modules optional for deployment.

Datasets

The proposed UMA-Net was evaluated on three diverse medical imaging datasets:

  • Lung Segmentation (Chest X-rays):
    • LN-JSRT: A simulated dataset derived from the JSRT dataset with a lower noise level.
  • HN-JSRT: A simulated dataset derived from the JSRT dataset with a higher noise level. These datasets were created by adding five types of simulated noise (erosion/dilation, shift, square reverse, polygonization, random erase) to expert-annotated ground truths.
  • Optic Disc and Cup Segmentation (Fundus Images):
    • RIGA Dataset: A publicly available dataset with annotations from multiple ophthalmologists, exhibiting inter-observer and intra-observer variations.
  • Breast DCE-MRI Segmentation:
    • Duke-Breast-Cancer-MRI Dataset: A dataset of 3D dynamic contrast-enhanced MRI scans. For this dataset, four different region-growing-based algorithms were used to generate pseudo-annotations, simulating systematic noise from automated methods. Expert annotations were used for testing.

Key Insights and Ideas

  • Leveraging Annotation Variance: The core insight is that the differences and agreements among multiple noisy annotations contain rich information about label reliability, which can be actively exploited rather than just averaged out.
  • Dual-Level Uncertainty Guidance: Combining pixel-level uncertainty (identifying unreliable pixels) with image-level quality assessment (identifying low-quality samples) provides a more comprehensive strategy for robust learning.
  • Auxiliary Predictor for Low-Quality Data: The idea of using an auxiliary predictor to learn from low-quality samples is a clever way to retain valuable feature representations without compromising the performance of the primary model, addressing the issue of discarding data.
  • Uncertainty-Weighted Losses: Adapting standard segmentation losses (Cross-Entropy and Dice) with pixel-wise uncertainty weights is a direct and effective mechanism for guiding the model towards reliable pixels.

Key Results

  • State-of-the-Art Performance: UMA-Net achieved state-of-the-art results on all three tested datasets (lung segmentation, optic disc/cup segmentation, and breast DCE-MRI segmentation).
  • Lung Segmentation: On the high-noise HN-JSRT dataset, UMA-Net achieved a Dice score of 88.75%, outperforming methods like PINT and Self-Calib by significant margins (5.79% and 3.36% respectively).
  • Optic Disc/Cup Segmentation: UMA-Net surpassed existing multi-source annotation methods (MR-Net, Self-Calib, AGNet, BEAL) by achieving a Dice score of 97.95% for optic disc and 90.70% for optic cup segmentation.
  • Breast DCE-MRI Segmentation: UMA-Net achieved a Dice score of 81.06% and IoU of 68.52%, outperforming other methods by at least 1.42% in Dice and 1.96% in Jaccard.
  • Ablation Studies: The results consistently showed that AUEM significantly improved upon baseline methods, and the addition of consistency loss and QAM with the auxiliary predictor further boosted performance. The auxiliary predictor was shown to be crucial for learning from low-quality samples.

Limitations and Future Work

The authors acknowledge a few limitations:

  • Fixed Number of Annotations: The current AUEM requires all images to have the same number of annotations. A more flexible solution would allow for adaptive handling of a varied number of annotations per sample.
  • Hard Assignment for Quality: The model currently uses a hard assignment strategy for feeding samples to either the primary or auxiliary predictor based on QAM's classification. A potential improvement would be a dynamic assignment strategy that uses quality-related weights to allow both predictors to learn from all images.

Future work will focus on addressing these limitations and further advancing multi-source annotation-based segmentation.

Summary updated

BibTeX
@article{wang2025learning,
  author = {Yifeng Wang and Luyang Luo and Mingxiang Wu and Qiong Wang and Hao Chen},
  title = {{Learning robust medical image segmentation from multi-source annotations}},
  journal = {Medical Image Analysis},
  year = {2025},
  doi = {10.1016/j.media.2025.103489},
}
Figure