Diffusion Deformable Model for 4D Temporal Medical Image Generation

Boah Kim, Jong Chul Ye

TL;DR

Researchers have developed a Diffusion Deformable Model (DDM) that generates realistic 4D (3D + time) medical images, such as a beating heart, by combining the realism of diffusion models with the structural integrity of image registration. By interpolating a "latent code" learned through a denoising process, the model can synthesize smooth, topology-preserving intermediate frames between two static 3D volumes.

What Problem Does This Paper Address?

Capturing the progression of anatomical changes—like a heart cycle or disease growth—is vital for diagnosis. However, obtaining high-resolution 4D MRI data is difficult because MRI requires long scan times, making it hard to capture rapid motion.

While AI can "fill in the gaps" by generating intermediate frames, existing methods have flaws:

  1. Generative Adversarial Networks (GANs) often create "hallucinations" or artificial features that don't exist in the patient.
  2. Standard Registration Methods (warping one image to another) often struggle to produce realistic intermediate steps, sometimes resulting in anatomically impossible transitions.

The authors aim to create a model that generates realistic, continuous temporal sequences while ensuring the underlying anatomy remains consistent and "connected" (topology preservation).

Methods

The authors propose the Diffusion Deformable Model (DDM), which splits the task into two specialized modules:

  • The Diffusion Module: Based on Denoising Diffusion Probabilistic Models (DDPM), this module learns to compare a "source" volume (e.g., end-diastole) and a "target" volume (e.g., end-systole). It produces a latent code ($c$) that contains detailed spatial information about how the source must change to become the target.
  • The Deformation Module: Instead of generating pixels from scratch, this module takes the latent code and the source image to estimate deformation fields ($\phi$). These fields act like a digital "stretching" instruction, warping the source image into the target shape using a Spatial Transformation Layer (STL).

The Secret Sauce: Once trained, you can scale the latent code by a factor ($\gamma$) between 0 and 1. Feeding a scaled code (e.g., $0.5c$) into the deformation module generates a perfectly intermediate frame. This allows for the generation of a continuous "geodesic path" of motion.

Datasets

The model was evaluated using the ACDC (Automated Cardiac Diagnosis Challenge) dataset, which contains 100 4D cardiac MRI scans.

  • Training/Testing: 90 scans for training, 10 for testing.
  • Preprocessing: Images were resampled to a consistent voxel spacing and cropped to $128 \times 128 \times 32$.

Key Insights and Ideas

  • Diffusion as a Feature Extractor: Traditionally, diffusion models are used to generate images from noise. Here, the authors use the diffusion "score function" to extract a highly informative latent representation of motion rather than just appearance.
  • Topology Preservation: Because the model uses a deformation module (warping) rather than pixel-wise generation, it is physically impossible for the model to "break" the anatomy or add random artifacts. The heart tissue stays connected.
  • Latent Scaling vs. Field Scaling: Unlike older methods that simply scale the final motion vectors (which can look robotic), scaling the latent code allows the model to capture more complex, non-linear anatomical dynamics.

Key Results

The DDM was compared against popular registration baselines like VoxelMorph (VM) and its diffeomorphic variant (VM-diff):

  • Image Quality: DDM achieved a PSNR of 30.725 dB, outperforming both VM (30.562) and VM-diff (29.481).
  • Anatomical Accuracy: Using Dice scores (a measure of how well the generated anatomy matches ground truth segmentation), DDM reached 0.802, a statistically significant improvement over the baselines.
  • Efficiency: While training takes about 6 hours, generating a new frame at inference time takes only 0.456 seconds, making it viable for real-time clinical visualization.

Limitations and Future Work

  • Dataset Size: The study was limited to the ACDC dataset (100 subjects). Testing on more diverse pathologies or different organs (like lungs) would be a necessary next step.
  • Source-Target Dependency: The model currently generates frames between two known endpoints. Future work could explore "extrapolating" motion into the future rather than just "interpolating" between two known states.
  • Hyper-parameter Sensitivity: The paper notes that the balance between the diffusion loss and deformation loss ($\lambda$) is critical; if the deformation loss is too low, the registration accuracy drops.

Summary updated

Abstract

Temporal volume images with 3D+t (4D) information are often used in medical imaging to statistically analyze temporal dynamics or capture disease progression. Although deep-learning-based generative models for natural images have been extensively studied, approaches for temporal medical image generation such as 4D cardiac volume data are limited. In this work, we present a novel deep learning model that generates intermediate temporal volumes between source and target volumes. Specifically, we propose a diffusion deformable model (DDM) by adapting the denoising diffusion probabilistic model that has recently been widely investigated for realistic image generation. Our proposed DDM is composed of the diffusion and the deformation modules so that DDM can learn spatial deformation information between the source and target volumes and provide a latent code for generating intermediate frames along a geodesic path. Once our model is trained, the latent code estimated from the diffusion module is simply interpolated and fed into the deformation module, which enables DDM to generate temporal frames along the continuous trajectory while preserving the topology of the source image. We demonstrate the proposed method with the 4D cardiac MR image generation between the diastolic and systolic phases for each subject. Compared to the existing deformation methods, our DDM achieves high performance on temporal volume generation.
BibTeX
@inproceedings{kim2022diffusion,
  author = {Boah Kim and Jong Chul Ye},
  title = {{Diffusion Deformable Model for 4D Temporal Medical Image Generation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  url = {https://openreview.net/forum?id=z-EcRRbejvg},
}

Loading related content...

Figure