Abstract
Cross-modality medical image segmentation is often hindered by the "domain shift" between imaging modalities, such as CT and MRI. This paper introduces DRL-STNet, a novel framework designed for Unsupervised Domain Adaptation (UDA). By integrating Disentangled Representation Learning (DRL) for high-fidelity image translation with an iterative self-training (ST) loop, the framework enables the transfer of knowledge from labeled source domains to unlabeled target domains without requiring paired data. DRL-STNet achieves a significant performance leap, outperforming previous state-of-the-art methods by 11.4% in Dice Similarity Coefficient (DSC) for abdominal organ segmentation.
Introduction
In medical artificial intelligence, a model trained on one imaging modality (e.g., CT) frequently fails when applied to another (e.g., MRI) because the visual "style" differs significantly despite identical underlying anatomy. This phenomenon, known as domain shift, typically requires the collection and manual annotation of thousands of new images for the target modality—a process that is both labor-intensive and expensive. Furthermore, obtaining paired datasets where the same patient receives both scans simultaneously is logistically challenging. DRL-STNet addresses these hurdles by providing a robust UDA framework that adapts models to new modalities using only unlabeled target data.
Methodology
The DRL-STNet framework operates through a two-phase pipeline designed to bridge the modality gap while preserving anatomical integrity.
1. Image Translation via Disentangled Representation Learning (DRL)
Traditional translation methods, such as CycleGAN, often struggle to maintain fine anatomical details. DRL-STNet utilizes a Generative Adversarial Network (GAN) to explicitly separate an image into two components: Content (anatomical structure) and Style (modality-specific appearance). By swapping the style of a labeled CT slice with the style of an unlabeled MRI slice, the framework generates "synthetic MRIs." These images possess the visual characteristics of real MRI scans while retaining the precise, ground-truth anatomical labels of the original CT scans.
2. Iterative Self-Training (ST)
To overcome the "reality gap"—where synthetic images may lack the subtle artifacts or textures of clinical scans—the framework employs an iterative self-training loop:
- Initial Training: A 3D segmentation model (nnU-Net) is trained on the synthetic MRI data.
- Pseudo-Labeling: The model generates predictions (pseudo-labels) for real, unlabeled MRI scans.
- Fine-Tuning: The model is iteratively retrained using a combination of synthetic data (with ground-truth labels) and real data (with the generated pseudo-labels). This allows the model to adapt to the specific noise patterns of real-world clinical data.
Experimental Setup
The framework was evaluated using a large-scale, multi-center dataset:
- Source Data: 2,050 abdominal CT scans from various benchmarks (TCIA, LiTS, MSD).
- Target Data: Over 4,000 unlabeled MRI scans.
- Evaluation: Testing was conducted on the FLARE challenge dataset, comprising 110 validation and 300 testing MRI scans across multiple sequences (T1, T2, DWI).
Results and Discussion
DRL-STNet established a new benchmark for cross-modality abdominal organ segmentation. Key performance metrics include:
- Overall Accuracy: The method outperformed previous state-of-the-art approaches (e.g., SIFA and CycleGAN) by 11.4% in DSC and 13.1% in Normalized Surface Dice (NSD).
- Organ-Specific Success: The model achieved high accuracy on large organs, such as the Liver (93.76% DSC) and Kidneys (~91% DSC).
- Clinical Viability: The average inference time is approximately 41 seconds per volume, suggesting potential for integration into clinical workflows.
Despite these gains, challenges remain in segmenting smaller or less distinct structures, such as the adrenal glands (49.78% DSC) and the gallbladder. Furthermore, the quality of the final segmentation remains dependent on the initial GAN-based translation; anatomical artifacts introduced during translation can propagate through the self-training loop.
Conclusion
DRL-STNet demonstrates that combining disentangled representation learning with iterative self-training is a highly effective strategy for unsupervised domain adaptation in medical imaging. By successfully bridging the gap between CT and MRI without the need for manual target labels, this framework paves the way for more scalable and robust medical AI. Future research will explore the use of Diffusion Models for higher-quality translation and multi-task learning to handle broader clinical scenarios.
Summary updated