Abstract: Imputation of missing images via source-to-target modality translation can improve diversity in medical imaging protocols. A pervasive approach for synthesizing target images involves one-shot mapping through generative adversarial networks (GAN). Yet, GAN models that implicitly characterize the image distribution can suffer from limited sample fidelity. Here, we propose a novel method based on adversarial diffusion modeling, SynDiff, for improved performance in medical image translation. To capture a direct correlate of the image distribution, SynDiff leverages a conditional diffusion process that progressively maps noise and source images onto the target image. For fast and accurate image sampling during inference, large diffusion steps are taken with adversarial projections in the reverse diffusion direction. To enable training on unpaired datasets, a cycle-consistent architecture is devised with coupled diffusive and non-diffusive modules that bilaterally translate between two modalities. Extensive assessments are reported on the utility of SynDiff against competing GAN and diffusion models in multi-contrast MRI and MRI-CT translation. Our demonstrations indicate that SynDiff offers quantitatively and qualitatively superior performance against competing baselines.
Introduction
In medical imaging, acquiring multiple modalities (e.g., T1-MRI, T2-MRI, and CT) for the same patient is ideal for diagnosis but often impractical due to high costs, scan time, or radiation exposure. Medical Image Translation aims to "synthesize" missing modalities from available ones. Existing solutions face a "trilemma": GANs are fast but often lack fine-grained structural fidelity; Diffusion Models (DMs) produce high detail but are slow; and most high-performing models require rare "paired" datasets. The proposed SynDiff framework addresses these challenges by creating a model that is fast, highly detailed, and functional on unpaired datasets.
Methods
SynDiff rests on three technical pillars to achieve efficient and high-fidelity translation:
- Adversarial Diffusion Process: Standard diffusion models take tiny steps to ensure noise remains Gaussian. SynDiff takes large steps ($k \gg 1$). To compensate for the "jump" in noise, it uses an Adversarial Projector—a generator-discriminator pair that ensures the denoised image at each step stays on the realistic data manifold.
- Source-Conditional Guidance: Unlike unconditional diffusion, SynDiff uses the source image (e.g., a T1 scan) to guide the reverse diffusion process at every step, ensuring the synthesized image maintains the patient's exact anatomy.
- Cycle-Consistent Architecture: To handle unpaired data, a two-stage "bilateral" approach is used. A non-diffusive module (GAN-like) creates an initial rough estimate, which then serves as the guidance for the diffusive module to refine the image. A cycle-consistency loss ensures that translating from A to B and back to A returns the original image.
Experiments and Results
The model was evaluated on three diverse datasets: the IXI dataset (healthy brain MRI), the BRATS dataset (glioma patient MRI), and a Pelvic MRI-CT dataset. While training was unsupervised (unpaired), registered paired volumes were used for the test set to quantitatively measure performance.
Quantitative Performance
SynDiff consistently outperformed state-of-the-art GANs (like CycleGAN, UNIT, and MUNIT) and standard diffusion models. On the IXI dataset, it improved Peak Signal-to-Noise Ratio (PSNR) by ~2.2dB over standard GANs. In the difficult MRI-to-CT translation task, SynDiff outperformed GANs by ~3.3dB in PSNR and ~14.4% in SSIM (Structural Similarity Index).
Efficiency and Quality
SynDiff achieves its best results with only 4 reverse steps, compared to the 1000+ steps required by standard models like DDPM. This results in an inference time roughly two orders of magnitude faster (e.g., 0.18s vs. 85s per slice). Visual inspections showed that SynDiff produced images with significantly lower noise and fewer artifacts, capturing fine anatomical details that GANs blurred or missed.
Discussion
While SynDiff offers significant improvements in fidelity and speed, training an adversarial diffusion model remains more computationally demanding than training a simple GAN. Furthermore, if the source modality fundamentally lacks the information needed to reconstruct the target, the problem remains "ill-posed." Future work may extend SynDiff to many-to-one translation and full 3D volumetric processing to further enhance its clinical utility.
Summary updated