SynthMix: Mixing up Aligned Synthesis for Medical Cross-Modality Domain Adaptation

Xinwen Zhang, Chaoyi Zhang, Dongnan Liu, Qianbi Yu, Tom Weidong Cai

Introduction

In the field of medical imaging, deep learning models often struggle with the "domain shift" problem. A model trained to segment organs in Magnetic Resonance Imaging (MRI) scans frequently fails when applied to Computed Tomography (CT) scans, despite the underlying anatomy being identical. Unsupervised Domain Adaptation (UDA) seeks to bridge this gap by using Generative Adversarial Networks (GANs) to translate images from a source modality to a target modality. However, standard GAN-based translation often lacks the fine-grained anatomical precision required for clinical applications. This paper introduces SynthMix, a novel training strategy designed to sharpen medical image translation through aligned mixup and local patch inspection.

The Problem: Lazy Discriminators and Structural Noise

Traditional UDA frameworks, such as CycleGAN or SIFA (Synergistic Image and Feature Adaptation), rely on global discriminators. These discriminators evaluate whether an entire image looks "realistic" for the target domain. This approach often leads to "lazy" evaluation: the generator learns to produce images that are globally convincing but lack the specific textural and contrast details necessary for high-precision segmentation.

Furthermore, standard data augmentation techniques like Mixup or CutMix—which blend two random images—often introduce "structural noise." When a model is forced to reconcile two different anatomical structures (e.g., two different hearts) while simultaneously learning a modality shift, the resulting synthetic images can become blurry or anatomically inconsistent.

The SynthMix Framework

SynthMix addresses these limitations by introducing a specialized training module that focuses on local consistency and content alignment. The framework consists of three primary components:

1. Aligned Mixup Synthesis

Unlike standard Mixup, which blends arbitrary image pairs, SynthMix utilizes content-aligned images. It blends a real source-domain image with its synthetic target-domain counterpart. Because these two images share the exact same anatomical structure, the model is not distracted by changing organ shapes. This allows the generator to focus exclusively on learning the subtle textural and contrast differences between modalities, effectively isolating the "style" of the target domain.

2. The Mixup Inspector ($I_{SynthMix}$)

To combat the issue of global-only evaluation, the authors introduce the Mixup Inspector. This specialized discriminator operates on a $k \times k$ grid rather than the whole image. When presented with a mixed image, the Inspector is tasked with identifying, on a patch-by-patch basis, which segments belong to the source domain and which belong to the target domain. This forces the generator to ensure that every local $32 \times 32$ pixel patch is indistinguishable from the real target domain.

3. Adversarial Training Policy

This setup creates a challenging "mini-max" game. The generator must produce synthetic images so realistic that the Inspector cannot distinguish them from real target patches, even when they are blended into a composite image. This prevents the common GAN artifact where a model produces a good-looking global image that fails upon closer inspection of local textures.

Experimental Evaluation

The researchers evaluated SynthMix on two major medical segmentation benchmarks:

  • MMWHS (Multi-Modality Whole Heart Segmentation): A dataset focused on cardiac structures, testing both MR-to-CT and CT-to-MR adaptation.
  • AOS (Abdominal Organ Segmentation): A robust dataset combining the CHAOS and Multi-Atlas Labeling challenges to test the model across diverse abdominal anatomies.

Key Results

SynthMix demonstrated significant performance improvements over existing state-of-the-art (SOTA) methods like CycleGAN and the baseline SIFA framework:

  • Cardiac Segmentation Accuracy: In MR-to-CT tasks, SynthMix achieved a Dice score of 82.5%, a substantial increase from the baseline SIFA score of 74.1%.
  • Boundary Precision: The Average Symmetric Surface Distance (ASSD), which measures boundary error, dropped from 7.0 to 3.8 on the MMWHS dataset, indicating much sharper organ boundaries.
  • Optimal Resolution: Through ablation studies, the authors determined that an $8 \times 8$ mask resolution provided the "sweet spot" for the Mixup Inspector, balancing local detail with global coherence.

Conclusion and Future Work

SynthMix proves that locality and content alignment are crucial for high-fidelity medical image translation. By moving beyond global discriminators and utilizing aligned mixup, the framework significantly reduces structural noise and improves segmentation accuracy across different modalities and organ types.

While the results are promising, future research could explore more flexible, object-aware masks rather than rigid $k \times k$ grids. Additionally, testing SynthMix as a plug-and-play module for other UDA architectures like UNIT or MUNIT would further validate its versatility in the broader field of medical computer vision.

Summary updated

Abstract

The adversarial methods showed advanced performance by producing synthetic images to mitigate the domain shift, a common problem due to the hardship of acquiring labelled data in medical field. Most existing studies focus on modifying the network architecture, but little has worked on the GAN training strategy. In this work, we propose SynthMix, an add-on module with a natural yet effective training policy that can promote synthetic quality without altering the network architecture. Following the adversarial philosophy of GAN, we designed a mix-up synthesis scheme termed SynthMix. It coherently mixed up aligned images of real and synthetic samples to stimulate the generation of fine-grained features, examined by an associated Inspector for the domain-specific details. We evaluated our method on two segmentation benchmarks among three publicly available datasets, where our method showed a significant performance gain compared with existing state-of-the-art approaches.
BibTeX
@inproceedings{zhang2023synthmix,
  author = {Xinwen Zhang and Chaoyi Zhang and Dongnan Liu and Qianbi Yu and Tom Weidong Cai},
  title = {{SynthMix: Mixing up Aligned Synthesis for Medical Cross-Modality Domain Adaptation}},
  booktitle = {ISBI},
  year = {2023},
  doi = {10.1109/isbi53787.2023.10230360},
}
Figure