A New Bidirectional Unsupervised Domain Adaptation Segmentation Framework

Munan Ning, Cheng Bian, Dong Wei, Shuang Yu, Chenglang Yuan, Yaohua Wang, Yang Guo, Kai Ma, Yefeng Zheng

TL;DR

This paper introduces BiUDA, a novel framework for bidirectional unsupervised domain adaptation in medical image segmentation. By leveraging disentangled representation learning and a unique "domain-aware" pattern encoder, the authors successfully bridge the performance gap that usually occurs when adapting models between different modalities (like MRI and CT) in both directions.

What Problem Does This Paper Address?

Deep learning models for medical imaging are notoriously sensitive to domain shift. A model trained to segment a heart in an MRI scan often fails miserably when applied to a CT scan because the "appearance" (intensity, noise, contrast) differs wildly, even if the underlying "content" (the anatomy) is the same.

Unsupervised Domain Adaptation (UDA) aims to fix this by adapting a model from a labeled source domain to an unlabeled target domain. However, the authors identify a major flaw in current SOTA methods: they are often "one-way streets." A method might work well for MRI $\to$ CT but perform poorly for CT $\to$ MRI. The authors call this phenomenon domain drop. In clinical reality, you might have labels for either modality, so a robust, bidirectional solution is essential.

Methods

The BiUDA framework is built on the foundation of Disentangled Representation Learning (DRPL). It splits an image into two distinct codes:

  1. Content Code ($c$): Represents domain-invariant features like anatomical structure and shape.
  2. Pattern Code ($p$): Represents domain-specific features like modality-specific appearance.

Key Innovations:

  • Domain-Aware Pattern Encoder ($E_p$): Unlike previous works that use two separate encoders for two domains, BiUDA uses a single, unified encoder. A "domain controller" (a simple 0 or 1 input) tells the encoder which modality it is looking at. This reduces the number of parameters and forces the encoder to explicitly learn the differences between domains.
  • Content-Pattern Consistency Loss ($\mathcal{L}^{cpc}$): To prevent the model from "hallucinating" or distorting anatomy during the translation process, this loss ensures that if you decompose an image and then recompose it, the resulting codes remain consistent with the originals.
  • Label Consistency Strategy: The framework creates "pseudo" training data by taking the anatomical content from a labeled source image and applying the pattern (style) of the target domain. This allows the segmenter to learn target-domain appearances while using the ground-truth labels from the source.

Datasets

The authors evaluated their framework on two challenging multi-modality public datasets:

  1. MMWHS (Multi-Modality Whole Heart Segmentation): 20 MRI and 20 CT cardiac scans. Targets include the ascending aorta, left atrium, and ventricles.
  2. MMAS (Multi-Modality Abdominal Segmentation): 20 MRI scans (from the CHAOS challenge) and 30 CT scans. Targets include the liver, kidneys, and spleen.

Key Insights and Ideas

  • The "Domain Drop" is Real: The paper quantifies how asymmetric current UDA methods are. Some methods see a performance drop twice as large when reversing the adaptation direction.
  • Unification over Duplication: Using a single domain-aware encoder is more than just an efficiency hack; it acts as a regularizer that helps the model understand the relationship between modalities better than two independent encoders would.
  • Anatomy is the Anchor: By strictly separating "what" is in the image (anatomy) from "how" it looks (modality), the model can perform segmentation on the content code alone, making it naturally robust to style changes.

Key Results

The framework was compared against several SOTA methods, including SIFA, ACE, and DISE. The primary metric used was the performance drop (the gap between the UDA model and a "gold standard" model trained directly on target labels).

  • MMWHS Dataset: BiUDA achieved a mean Dice performance drop of only 8.49%, significantly lower than the next best method, ACE (13.47%), and much better than the popular SIFA (21.84%).
  • MMAS Dataset: BiUDA led with a mean Dice drop of 4.39%, outperforming ACE (5.27%) and SIFA (7.52%).
  • Bidirectional Stability: As shown in the paper's visualizations, the gap between "Forward" (MRI $\to$ CT) and "Backward" (CT $\to$ MRI) performance is much narrower in BiUDA than in competing frameworks, proving its reliability regardless of which modality is labeled.

Limitations and Future Work

While the paper demonstrates impressive results, a few areas remain for exploration:

  • 2D vs. 3D: The current framework operates on 2D slices. While standard for many UDA tasks, medical data is inherently 3D, and extending this to 3D volumes could further improve anatomical consistency.
  • Computational Overhead: The framework involves multiple encoders, a generator, a segmenter, and a discriminator. While the unified encoder helps, the training process is likely more complex than simpler alignment-based methods.
  • Generalization: The authors suggest that while they focused on MRI and CT, the framework could theoretically handle other modalities (like PET or Ultrasound), which remains a promising direction for future research.

Summary updated

Abstract

Domain shift happens in cross-domain scenarios commonly because of the wide gaps between different domains: when applying a deep learning model well-trained in one domain to another target domain, the model usually performs poorly. To tackle this problem, unsupervised domain adaptation (UDA) techniques are proposed to bridge the gap between different domains, for the purpose of improving model performance without annotation in the target domain. Particularly, UDA has a great value for multimodal medical image analysis, where annotation difficulty is a practical concern. However, most existing UDA methods can only achieve satisfactory improvements in one adaptation direction (e.g., MRI to CT), but often perform poorly in the other (CT to MRI), limiting their practical usage. In this paper, we propose a bidirectional UDA (BiUDA) framework based on disentangled representation learning for equally competent two-way UDA performances. This framework employs a unified domain-aware pattern encoder which not only can adaptively encode images in different domains through a domain controller, but also improve model efficiency by eliminating redundant parameters. Furthermore, to avoid distortion of contents and patterns of input images during the adaptation process, a content-pattern consistency loss is introduced. Additionally, for better UDA segmentation performance, a label consistency strategy is proposed to provide extra supervision by recomposing target-domain-styled images and corresponding source-domain annotations. Comparison experiments and ablation studies conducted on two public datasets demonstrate the superiority of our BiUDA framework to current state-of-the-art UDA methods and the effectiveness of its novel designs. By successfully addressing two-way adaptations, our BiUDA framework offers a flexible solution of UDA techniques to the real-world scenario.
BibTeX
@inproceedings{ning2021new,
  author = {Munan Ning and Cheng Bian and Dong Wei and Shuang Yu and Chenglang Yuan and Yaohua Wang and Yang Guo and Kai Ma and Yefeng Zheng},
  title = {{A New Bidirectional Unsupervised Domain Adaptation Segmentation Framework}},
  booktitle = {IPMI},
  year = {2021},
  doi = {10.1007/978-3-030-78191-0\_38},
}

Loading related content...

Figure