Unsupervised Medical Image Translation With Adversarial Diffusion Models

Muzaffer Özbey, Onat Dalmaz, Salman Ul Hassan Dar, Hasan Atakan Bedel, Saban Öztürk, Alper Güngör, Tolga Çukur

TL;DR

SynDiff is a novel unsupervised framework for medical image translation that combines the high-fidelity detail of diffusion models with the speed and unpaired learning capabilities of Generative Adversarial Networks (GANs). By using a "source-conditional adversarial projector," it can synthesize high-quality medical images (like T2-weighted MRI from T1-weighted MRI) in just a few steps, outperforming traditional GANs and standard diffusion models in both accuracy and efficiency.

What Problem Does This Paper Address?

In medical imaging, acquiring multiple modalities (e.g., T1-MRI, T2-MRI, and CT) for the same patient is ideal for diagnosis but often impractical due to high costs, scan time, or radiation exposure. Medical Image Translation aims to "synthesize" missing modalities from available ones.

Existing solutions face a "trilemma":

  1. GANs are fast but often suffer from "mode collapse" or lack fine-grained structural fidelity.
  2. Diffusion Models (DMs) produce incredible detail and diversity but are notoriously slow, requiring thousands of iterative steps.
  3. Supervision Requirements: Most high-performing models require "paired" data (perfectly aligned images of the same patient in different modalities), which is rare in clinical practice.

The authors tackle these issues by creating a model that is fast, highly detailed, and works on unpaired datasets.

Methods

The authors introduce SynDiff, which rests on three technical pillars:

  • Adversarial Diffusion Process: Standard diffusion models take tiny steps to ensure the noise remains Gaussian. SynDiff takes large steps ($k \gg 1$). To compensate for the "jump" in noise, it uses an Adversarial Projector—a generator-discriminator pair that ensures the denoised image at each step stays on the realistic data manifold.
  • Source-Conditional Guidance: Unlike unconditional diffusion, SynDiff uses the "source" image (e.g., a T1 scan) to guide the reverse diffusion process at every step. This ensures the synthesized image (e.g., a T2 scan) maintains the exact anatomy of the patient.
  • Cycle-Consistent Architecture: To handle unpaired data, the authors use a two-stage "bilateral" approach. A non-diffusive module (standard GAN-like) creates an initial rough estimate of the translation. This estimate then serves as the "ground truth" guidance for the diffusive module, which refines the image. A cycle-consistency loss ensures that translating from A to B and back to A returns the original image.

Datasets

The model was evaluated on three diverse datasets:

  1. IXI Dataset: Healthy brain MRI scans (T1, T2, and PD-weighted).
  2. BRATS Dataset: Brain MRI scans from glioma patients (T1, T2, and FLAIR), which includes pathological tissues.
  3. Pelvic MRI-CT Dataset: Cross-modal translation between MRI and CT scans, a significantly harder task due to the different physical principles of the two modalities.

While training was unsupervised (unpaired), the authors used registered, paired volumes for the test set to quantitatively measure performance.

Key Insights and Ideas

  • The "Aha" Moment: You don't need 1000 steps if you have a discriminator. By treating each large reverse diffusion step as a conditional GAN task, you get the stability and likelihood-based benefits of diffusion with the speed of GANs.
  • Anatomy Preservation: By feeding the source image into the generator at every diffusion step, the model treats the source as a spatial anchor, preventing the "hallucinations" common in standard GANs.
  • Efficiency: SynDiff achieves its best results with only 4 reverse steps, compared to the 1000+ steps required by standard models like DDPM. This makes it viable for real-time clinical applications.

Key Results

  • Quantitative Superiority: SynDiff consistently outperformed state-of-the-art GANs (like CycleGAN/cGAN, UNIT, and MUNIT) and standard diffusion models. For example, on the IXI dataset, it improved Peak Signal-to-Noise Ratio (PSNR) by ~2.2dB over standard GANs.
  • Cross-Modal Success: In the difficult MRI-to-CT translation task, SynDiff outperformed GANs by a massive ~3.3dB in PSNR and ~14.4% in SSIM (Structural Similarity Index).
  • Speedup: SynDiff's inference time is roughly two orders of magnitude faster than standard diffusion models (e.g., 0.18 seconds vs. 85 seconds per slice).
  • Qualitative Quality: Visual inspections showed that SynDiff produced images with significantly lower noise and fewer artifacts, capturing fine anatomical details that GANs blurred or missed.

Limitations and Future Work

  • Training Complexity: While inference is fast, training an adversarial diffusion model is more computationally demanding than training a simple GAN.
  • Information Bottleneck: The authors note that if the source modality fundamentally lacks the information needed to reconstruct the target (e.g., certain tissue contrasts), the problem remains "ill-posed," and even SynDiff cannot perfectly recover the missing data.
  • Future Directions: The authors suggest extending SynDiff to many-to-one translation (using multiple source modalities to synthesize one target) and moving from 2D slice-based processing to full 3D volumetric translation.

Summary updated

Abstract

Imputation of missing images via source-to-target modality translation can improve diversity in medical imaging protocols. A pervasive approach for synthesizing target images involves one-shot mapping through generative adversarial networks (GAN). Yet, GAN models that implicitly characterize the image distribution can suffer from limited sample fidelity. Here, we propose a novel method based on adversarial diffusion modeling, SynDiff, for improved performance in medical image translation. To capture a direct correlate of the image distribution, SynDiff leverages a conditional diffusion process that progressively maps noise and source images onto the target image. For fast and accurate image sampling during inference, large diffusion steps are taken with adversarial projections in the reverse diffusion direction. To enable training on unpaired datasets, a cycle-consistent architecture is devised with coupled diffusive and non-diffusive modules that bilaterally translate between two modalities. Extensive assessments are reported on the utility of SynDiff against competing GAN and diffusion models in multi-contrast MRI and MRI-CT translation. Our demonstrations indicate that SynDiff offers quantitatively and qualitatively superior performance against competing baselines.
BibTeX
@article{ozbey2023unsupervised,
  author = {Muzaffer Özbey and Onat Dalmaz and Salman Ul Hassan Dar and Hasan Atakan Bedel and Saban Öztürk and Alper Güngör and Tolga Çukur},
  title = {{Unsupervised Medical Image Translation With Adversarial Diffusion Models}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2023},
  url = {https://openreview.net/forum?id=olweLudY9T},
}

Loading related content...

Figure