Domain-Agnostic Learning with Anatomy-Consistent Embedding for Cross-Modality Liver Segmentation

Junlin Yang, Nicha C. Dvornek, Fan Zhang, Juntang Zhuang, Julius Chapiro, Ming De Lin, James S. Duncan

TL;DR

This paper introduces DALACE, a novel framework for Domain-Agnostic Learning (DAL) in medical image segmentation. DALACE learns anatomy-consistent embeddings that are invariant to different imaging modalities (like CT and MRI), enabling robust liver segmentation across diverse and unseen datasets, outperforming existing methods in both standard Domain Adaptation (DA) and the more challenging DAL tasks.

What Problem Does This Paper Address?

Deep learning models excel at image segmentation but typically require large, labeled datasets for each specific task and domain. In medical imaging, acquiring such data is expensive and time-consuming, especially when dealing with variations in scanners, protocols, and imaging modalities (e.g., CT vs. MRI). While Domain Adaptation (DA) aims to bridge the gap between a source and a specific target domain, this paper tackles a more ambitious problem: Domain-Agnostic Learning (DAL). DAL seeks to transfer knowledge from a labeled source domain to multiple, heterogeneous, and unknown target domains. This is crucial because medical data often comes from various sources, and a single model needs to generalize across them without prior knowledge of each target domain's characteristics. Specifically, the paper focuses on cross-modality liver segmentation, where CT and MRI provide complementary information but have significant visual differences.

Methods

The core of the proposed solution is the Domain-Agnostic Learning framework with Anatomy-Consistent Embedding (DALACE). DALACE is an end-to-end trainable model that disentangles anatomical information from modality-specific information. It achieves this through:

  • Anatomy Encoders and Modality Encoders: These components extract anatomical features and modality-specific codes from input images.
  • Style-Based Generators: These generators reconstruct images, either self-reconstructing the original modality or cross-reconstructing into another modality.
  • Anatomy Preserving Module (APM): This module uses adversarial training to ensure that the learned anatomical embeddings preserve high-level semantic information, making them consistent across modalities. It leverages a segmentation module (U-Net) and a discriminator to enforce this.
  • Domain Agnostic Module (DAM): This module also uses adversarial training to make the learned embeddings invariant to the domain. It employs discriminators to distinguish between real and generated images, pushing the modality information into modality codes and forcing the anatomy embeddings to be domain-agnostic.

By disentangling anatomy and modality, DALACE learns a representation that is useful for segmentation regardless of the input image's origin.

Datasets

The study uses the following datasets:

  • CT Data: 130 CT scans from the LiTS challenge (ISBI and MICCAI 2017) with ground truth liver segmentation masks. This serves as the primary labeled source domain.
  • MR Data: Multi-phasic MR scans from 20 patients, including pre-contrast, 20s post-contrast (arterial phase), and 70s post-contrast (venous phase) MR images. This data is unlabeled for the DA and DAL tasks. Manual liver segmentations were created by a radiology expert for evaluation.

The data was normalized, resliced to be isotropic, and underwent bias field correction for MR data. Experiments were conducted using 5-fold cross-validation.

Key Insights and Ideas

  • DAL is a critical next step beyond DA: The paper highlights that while DA is useful, real-world medical imaging scenarios often involve multiple, unknown target domains, making DAL a more practical and challenging problem.
  • Disentanglement is key for generalization: By separating anatomical content from modality style, DALACE learns representations that are robust to domain shifts and can generalize to unseen modalities and domains.
  • Anatomy consistency is crucial for segmentation: Simply aligning marginal distributions between domains isn't enough. Enforcing anatomical consistency ensures that the learned features are semantically meaningful for the segmentation task.
  • Interpretability through disentanglement: Visualizing the disentangled representations (domain-agnostic images) provides insights into what the model has learned, making it more transparent.
  • Synergy in Joint Learning: Combining labeled CT and unlabeled MR data within the DALACE framework for joint learning yields superior results, demonstrating the benefit of integrating information from different modalities.

Key Results

  • Domain Adaptation (DA): DALACE achieved a Dice Similarity Coefficient (DSC) of 0.847, significantly outperforming CycleGAN (0.721), TD-GAN (0.793), and DADR (0.806) on the task of adapting from CT to pre-contrast MR.
  • Domain-Agnostic Learning (DAL): For the more challenging DAL task (adapting from CT to heterogeneous multi-phasic MR), DALACE achieved a DSC of 0.794, a substantial improvement over CycleGAN (0.522), TD-GAN (0.719), and DADR (0.742).
  • Joint Learning: In a joint learning setting (using both labeled CT and MR), DALACE achieved DSC scores of 0.911 on CT and 0.907 on MR, outperforming other methods and even supervised U-Nets trained separately on each modality.
  • Ablation Studies: Removing the disentanglement component (effectively using a TD-GAN-like approach) led to a performance drop from 0.847 to 0.793 for DA and 0.794 to 0.719 for DAL, underscoring the benefits of disentanglement.

Limitations and Future Work

  • Style Transfer Evaluation: While DALACE can perform style transfer, quantitatively evaluating its results is challenging without paired CT and MR ground truth for this specific task.
  • Integration of Modalities: The paper suggests that further exploration into integrating information from different modalities, beyond what was done in joint learning, is a promising direction.
  • Generalizability to Other Tasks: The paper focuses on liver segmentation. Future work could explore the application of DALACE to other medical imaging segmentation tasks or even classification tasks.
  • Computational Cost: While not explicitly detailed as a limitation, end-to-end training of such complex models can be computationally intensive.

The authors envision DALACE as a step towards better integration of different medical imaging modalities, improving clinical workflows and addressing data scarcity.

Summary updated

Abstract

Domain Adaptation (DA) has the potential to greatly help the generalization of deep learning models. However, the current literature usually assumes to transfer the knowledge from the source domain to a specific known target domain. Domain Agnostic Learning (DAL) proposes a new task of transferring knowledge from the source domain to data from multiple heterogeneous target domains. In this work, we propose the Domain-Agnostic Learning framework with Anatomy-Consistent Embedding (DALACE) that works on both domain-transfer and task-transfer to learn a disentangled representation, aiming to not only be invariant to different modalities but also preserve anatomical structures for the DA and DAL tasks in cross-modality liver segmentation. We validated and compared our model with state-of-the-art methods, including CycleGAN, Task Driven Generative Adversarial Network (TD-GAN), and Domain Adaptation via Disentangled Representations (DADR). For the DA task, our DALACE model outperformed CycleGAN, TD-GAN ,and DADR with DSC of 0.847 compared to 0.721, 0.793 and 0.806. For the DAL task, our model improved the performance with DSC of 0.794 from 0.522, 0.719 and 0.742 by CycleGAN, TD-GAN, and DADR. Further, we visualized the success of disentanglement, which added human interpretability of the learned meaningful representations. Through ablation analysis, we specifically showed the concrete benefits of disentanglement for downstream tasks and the role of supervision for better disentangled representation with segmentation consistency to be invariant to domains with the proposed Domain-Agnostic Module (DAM) and to preserve anatomical information with the proposed Anatomy-Preserving Module (APM).
BibTeX
@inproceedings{yang2019domain,
  author = {Junlin Yang and Nicha C. Dvornek and Fan Zhang and Juntang Zhuang and Julius Chapiro and Ming De Lin and James S. Duncan},
  title = {{Domain-Agnostic Learning with Anatomy-Consistent Embedding for Cross-Modality Liver Segmentation}},
  booktitle = {Workshops @ ICCV},
  year = {2019},
  url = {https://openreview.net/forum?id=XOu-SUE5w3W},
}
Figure