Unsupervised Domain Adaptation with Dual-Scheme Fusion Network for Medical Image Segmentation

Danbing Zou, Qikui Zhu, Pingkun YanIJCAI 2020 · DOI

TL;DR

This paper introduces a novel Dual-Scheme Fusion Network (DSFN) for unsupervised domain adaptation in medical image segmentation. By simultaneously learning to translate images in both source-to-target and target-to-source directions and fusing the resulting segmentations, DSFN effectively bridges domain gaps, leading to significantly improved segmentation accuracy without requiring target domain labels.

What Problem Does This Paper Address?

Deep learning models excel at image segmentation, but their performance often degrades when applied to data from different domains (e.g., different scanners, protocols, or patient populations). This "domain shift" problem necessitates retraining models with large amounts of labeled target domain data, which is often expensive and time-consuming to acquire, especially in medical imaging. Unsupervised domain adaptation (UDA) aims to overcome this by leveraging labeled source data and unlabeled target data. Existing UDA methods often focus on either translating source images to look like target images or vice-versa, which can lead to imbalanced information flow and fail to fully eliminate the domain gap. This paper addresses the limitation of these one-way translation approaches by proposing a more balanced, dual-directional adaptation strategy.

Methods

The core of the proposed method is the Dual-Scheme Fusion Network (DSFN), built upon the CycleGAN architecture. DSFN integrates image translation and segmentation into a unified framework.

  • Dual-Scheme Fusion: Instead of just source-to-target or target-to-source translation, DSFN learns both simultaneously. This creates a "balanced joint information flow" to reduce the domain gap.
  • Joint Translation-Segmentation Module: The generator in the CycleGAN framework is modified to have a shared encoder for both image translation and segmentation branches. This parameter sharing not only reduces computational cost but also allows the two tasks to mutually benefit.
  • Cycle Training: The network is trained in a cyclic manner, meaning the backward pass (target-to-source) is also used for testing.
  • Fusion Strategy: For a given target image, DSFN produces two segmentation predictions: one from the target-to-source translation path and another from the source-to-target path. These predictions are then fused (by averaging probabilities) to obtain a more robust final segmentation.
  • Feature Adaptation Loss: To further bridge the feature-level gap between domains, an adversarial loss is introduced to ensure that the feature representations of source and target-like images are indistinguishable.

Datasets

The proposed method is evaluated on two medical imaging datasets:

  • BRATS dataset: Used for multi-modality brain tumor segmentation. The source domain is T2 MRI images with whole tumor annotations, and the target domains are T1, T1CE, and FLAIR MRI images without annotations.
  • MM-WHS dataset: Used for cardiac structure segmentation. The source domain consists of MR images with annotations for four heart structures, and the target domain is CT images of the same structures without annotations.

Key Insights and Ideas

  • Symmetry is Key: The paper's central insight is that a balanced, bidirectional approach (source-to-target and target-to-source) is more effective at closing the domain gap than unidirectional methods.
  • Synergy between Translation and Segmentation: Sharing an encoder between translation and segmentation tasks allows them to inform and improve each other, leading to better overall performance.
  • Fusion for Robustness: Combining segmentations from different adaptation perspectives yields more accurate and robust results, smoothing out errors and improving boundary adherence.
  • Feature-Level Alignment Matters: Incorporating feature adaptation loss further enhances performance by ensuring that the underlying representations are domain-invariant.

Key Results

  • Significant Performance Improvement: DSFN achieved substantial improvements in both Dice Score and Hausdorff Distance compared to baseline methods (no domain adaptation) and other state-of-the-art UDA techniques on both BRATS and MM-WHS datasets.
  • BRATS Dataset: Improved Dice Score from 22.6% to 66.1% for tumor segmentation and reduced Hausdorff Distance from 46.8mm to 15.6mm.
  • MM-WHS Dataset: Improved Dice Score from 17.2% to 75.8% for cardiac segmentation and reduced Hausdorff Distance from 44.8mm to 11.4mm.
  • Outperforming SIFA: DSFN outperformed SIFA (a strong source-to-target method) by increasing Dice Score by 6.8% on BRATS and 2.8% on MM-WHS, demonstrating the benefit of the dual-scheme approach.
  • Ablation Studies: Confirmed the effectiveness of both the fusion strategy and feature adaptation loss, with feature adaptation showing particularly strong gains on the MM-WHS dataset.

Limitations and Future Work

  • Computational Cost: While parameter sharing helps, the dual-scheme approach inherently involves more computation than single-scheme methods.
  • Fusion Strategy: The paper mentions averaging probabilities as the fusion strategy, but other fusion techniques could be explored for further optimization.
  • Generalizability: While tested on two medical imaging tasks, further validation on a wider range of medical imaging modalities and segmentation tasks would be beneficial.
  • Future Work: The authors suggest exploring more advanced fusion strategies and potentially incorporating other domain adaptation techniques to further enhance performance.

Summary updated

BibTeX
@inproceedings{zou2020unsupervised,
  author = {Danbing Zou and Qikui Zhu and Pingkun Yan},
  title = {{Unsupervised Domain Adaptation with Dual-Scheme Fusion Network for Medical Image Segmentation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/455},
}
Figure