Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment

Wei Feng, Lie Ju, Lin Wang, Kaimin Song, Xin Zhao, Zongyuan GeAAAI 2023 · OpenReview

TL;DR

This paper introduces a novel unsupervised domain adaptation (UDA) framework for medical image segmentation that tackles the issue of overconfident, erroneous predictions on unseen target data. By leveraging selective entropy constraints and adaptive semantic alignment, the method significantly improves segmentation accuracy across different medical imaging modalities.

What Problem Does This Paper Address?

Deep learning models excel at medical image segmentation when trained on data from a specific source domain. However, their performance drastically degrades when applied to data from a different domain (e.g., images from a different scanner or protocol) due to distribution shifts. This is a major hurdle for deploying AI in real-world clinical settings, as acquiring and annotating large datasets for every new domain is prohibitively expensive and time-consuming. While existing UDA methods use techniques like adversarial training to bridge this domain gap, they often produce "overconfident" predictions – the model is very sure about its incorrect segmentation. This paper aims to address this overconfidence issue and improve the robustness of UDA for medical image segmentation.

Methods

The proposed framework, "Selective Entropy Constraints and Adaptive Semantic Alignment" (SE-ASA), introduces two key innovations:

  1. Selective Entropy Constraints:
    • Data Augmentation: Two data augmentation strategies (random transformations and Fourier-based augmentation) are used to create multiple versions of target images.
  2. Pixel Reliability Identification: The model's prediction consistency across these augmented images is used to identify "reliable" pixels (where predictions are consistent) and "unreliable" pixels (where predictions vary).
  3. Confidence Remolding: For reliable pixels, the model's prediction entropy is minimized to boost confidence. For unreliable pixels, entropy is maximized to reduce overconfidence and prevent them from unduly influencing the model.
  4. Adaptive Semantic Alignment:
    • Prototype-Based Alignment: This module aims to align feature distributions between the source and target domains at a class level. It calculates "prototypes" (representative feature vectors) for each class in both domains.
  5. Unreliable Pixel Filtering: Crucially, unreliable pixels identified in the previous step are excluded when calculating target domain prototypes. This prevents noisy or incorrect predictions from corrupting the semantic alignment process.
  6. Distance Minimization: The distance between prototypes of the same class from the source and target domains is minimized.

These components are integrated into an end-to-end training pipeline that also includes standard adversarial training for global distribution adaptation.

Datasets

The experiments were conducted on the Multi-Modality Whole Heart Segmentation (MMWHS) Challenge 2017 dataset. This dataset contains unpaired CT and MRI volumes, making it suitable for cross-modality domain adaptation tasks. The researchers performed adaptation in both directions: CT to MRI and MRI to CT. They used pre-processed data released by the SIFA V2 method for consistency.

Key Insights and Ideas

  • Overconfidence is a critical problem in UDA for medical imaging: The paper highlights that even state-of-the-art UDA methods can produce highly confident but wrong predictions on target data, a problem exacerbated by domain shifts.
  • Pixel-level reliability is key: Identifying which pixels are reliably segmented versus which are not is crucial for both confidence regularization and semantic alignment.
  • Selective confidence adjustment is more effective: Instead of a blanket approach, tailoring confidence adjustments (minimizing entropy for reliable, maximizing for unreliable pixels) leads to better results.
  • Filtering unreliable pixels for prototype calculation is vital: Using noisy pseudo-labels or inconsistent predictions to compute class prototypes can harm semantic alignment. Removing these unreliable pixels leads to more accurate prototype alignment.
  • Complementary data augmentation: The two proposed augmentation methods (random transformations and Fourier-based) work together to improve pixel reliability assessment.

Key Results

The proposed SE-ASA method significantly outperformed state-of-the-art comparison algorithms on the cardiac structure segmentation task.

  • Cardiac CT → Cardiac MRI: The method achieved an average Dice score of 80.9%, a substantial improvement over the best comparison algorithm (CRST at 75.5%). It also showed a significant reduction in average Symmetric Surface Distance (ASD).
  • Cardiac MRI → Cardiac CT: The method achieved an average Dice score of 80.9%, again outperforming CRST (75.5%).
  • Ablation studies confirmed the effectiveness of both the selective entropy constraints (improving Dice by 3.2% over adversarial training alone) and adaptive semantic alignment (improving Dice by 2.6%). Removing unreliable pixels for semantic alignment improved the Dice score by 3.4%.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their method significantly reduces overconfidence, it might not entirely eliminate it. The reliance on pseudo-labels for target domain training inherently carries some risk of error propagation.
  • Future Work: The authors suggest that their approach is effective and generalizable to other unsupervised medical image segmentation tasks. The framework's ability to identify reliable and unreliable pixels and adapt confidence selectively opens avenues for further research in robust UDA for medical imaging.

Summary updated

Abstract

Generalizing a deep learning model to new domains is crucial for computer-aided medical diagnosis systems. Most existing unsupervised domain adaptation methods have made significant progress in reducing the domain distribution gap through adversarial training. However, these methods may still produce overconfident but erroneous results on unseen target images. This paper proposes a new unsupervised domain adaptation framework for cross-modality medical image segmentation. Specifically, We first introduce two data augmentation approaches to generate two sets of semantics-preserving augmented images. Based on the model's predictive consistency on these two sets of augmented images, we identify reliable and unreliable pixels. We then perform a selective entropy constraint: we minimize the entropy of reliable pixels to increase their confidence while maximizing the entropy of unreliable pixels to reduce their confidence. Based on the identified reliable and unreliable pixels, we further propose an adaptive semantic alignment module which performs class-level distribution adaptation by minimizing the distance between same class prototypes between domains, where unreliable pixels are removed to derive more accurate prototypes. We have conducted extensive experiments on the cross-modality cardiac structure segmentation task. The experimental results show that the proposed method significantly outperforms the state-of-the-art comparison algorithms. Our code and data are available at https://github.com/fengweie/SE_ASA.
BibTeX
@inproceedings{feng2023unsupervised,
  author = {Wei Feng and Lie Ju and Lin Wang and Kaimin Song and Xin Zhao and Zongyuan Ge},
  title = {{Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=2T1OCQrzl6G},
}
Figure