Abstract
Deep learning models trained on medical images from a source domain (e.g. imaging modality) often fail when deployed on images from a different target domain, despite imaging common anatomical structures. Deep unsupervised domain adaptation (UDA) aims to improve the performance of a deep neural network model on a target domain, using solely unlabelled target domain data and labelled source domain data. However, current state-of-the-art methods exhibit reduced performance when target data is scarce. In this work, we introduce a new data efficient UDA method for multi-domain medical image segmentation. The proposed method combines a novel VAE-based feature prior matching, which is data-efficient, and domain adversarial training to learn a shared domain-invariant latent space which is exploited during segmentation. Our method is evaluated on a public multi-modality cardiac image segmentation dataset by adapting from the labelled source domain (3D MRI) to the unlabelled target domain (3D CT). We show that by using only one single unlabelled 3D CT scan, the proposed architecture outperforms the state-of-the-art in the same setting. Finally, we perform ablation studies on prior matching and domain adversarial training to shed light on the theoretical grounding of the proposed method.
Introduction
Deep learning models are notoriously sensitive to "domain shift." A model trained to segment organs in MRI scans often fails completely when applied to CT scans, even though the underlying anatomy is the same. While Unsupervised Domain Adaptation (UDA) aims to fix this by aligning the "source" (MRI) and "target" (CT) domains without needing new labels, most current methods rely on Generative Adversarial Networks (GANs). GANs are data-hungry; if target data is scarce—a common reality in clinical practice—GAN-based alignment often collapses or produces poor results. This paper introduces a hybrid architecture that merges the generative power of Variational Autoencoders (VAEs) with the discriminative alignment of adversarial training to address this challenge.
Methodology
The proposed method focuses on learning a domain-invariant latent space $Z$ that can be exploited for segmentation across different modalities.
Shared Latent Space and VAE-based Prior Matching
The model uses domain-specific encoders to project both MRI and CT images into a shared latent space. The core innovation is VAE-based Prior Matching. In a standard VAE, the latent space is regularized using KL-divergence to match a Gaussian prior ($\mathcal{N}(0, I)$). Because this KL-divergence can be calculated analytically, it acts as a powerful, data-efficient constraint. It forces the feature distributions of both domains to overlap at a central "prior," even if the model has only seen a few target samples.
Domain Adversarial Training and Task Consistency
To further refine the alignment, a discriminator ($D_{isc}$) tries to distinguish between reconstructed source images and "fake" source images generated from target features. Additionally, a task-consistency loss encourages the model to produce consistent segmentations even when features are passed through a "cross-domain" cycle. By performing segmentation in the VAE's latent space rather than on reconstructed images, the model focuses on semantic features rather than pixel-level style, making it more robust to modality-specific noise.
Experiments and Results
The method was evaluated on the MICCAI Multi-modality Whole Heart Segmentation (MM-WHS) Challenge dataset, adapting from 20 labelled 3D MRI scans to unlabelled 3D CT scans. The task involved segmenting four structures: left ventricle myocardium (LV-M), left atrium blood cavity (LA-B), left ventricle blood cavity (LV-B), and ascending aorta (A-A).
Quantitative Evaluation
The most significant finding is the performance in the "one-scan" scenario. As shown in the results table, the proposed method achieved a 72.18% Mean Dice score using only a single unlabelled CT scan. This significantly outperforms the state-of-the-art SIFA-1 (62.51%) and Pnp-AdaNet-1 (42.19%) models in the same setting.
Remarkably, the proposed method using just one scan performed nearly as well as the SIFA model trained on sixteen scans (75.77%), representing a 16x improvement in data efficiency.
Discussion and Conclusion
The efficiency of the proposed method stems from replacing the "guessing" of target distributions—typical of adversarial UDA—with a fixed mathematical target (the Gaussian prior), which does not require a large sample size to define.
While the model aligns global distributions well, it does not explicitly enforce anatomical constraints (e.g., connectivity). Furthermore, experiments with extremely limited data (three 2D slices) showed signs of overfitting, suggesting a lower bound on data requirements. Future work will explore "task-driven" generative modeling to further improve the reliability of the latent space for downstream clinical use.
Summary updated