Accelerating Diffusion Models via Pre-segmentation Diffusion Sampling for Medical Image Segmentation

Xutao Guo, Yanwu Yang, Chenfei Ye, Shang Lu, Yang Xiang, Ting Ma

Abstract

Based on the Denoising Diffusion Probabilistic Model (DDPM), medical image segmentation can be described as a conditional image generation task, which allows to compute pixel-wise uncertainty maps of the segmentation and allows an implicit ensemble of segmentations to boost the segmentation performance. However, DDPM requires many iterative denoising steps to generate segmentations from Gaussian noise, resulting in extremely inefficient inference. To mitigate the issue, we propose a principled acceleration strategy, called pre-segmentation diffusion sampling DDPM (PD-DDPM), which is specially used for medical image segmentation. The key idea is to obtain pre-segmentation results based on a separately trained segmentation network, and construct noise predictions (non-Gaussian distribution) according to the forward diffusion rule. We can then start with noisy predictions and use fewer reverse steps to generate segmentation results. Experiments show that PD-DDPM yields better segmentation results over representative baseline methods even if the number of reverse steps is significantly reduced. Moreover, PD-DDPM is orthogonal to existing advanced segmentation models, which can be combined to further improve the segmentation performance.

Introduction

Denoising Diffusion Probabilistic Models (DDPMs) have emerged as a powerful tool in medical imaging due to their probabilistic nature. Unlike traditional deterministic models, DDPMs can generate multiple segmentation hypotheses, allowing clinicians to visualize uncertainty maps. These maps highlight regions where the model is less confident, providing a crucial safety layer to avoid misdiagnosis.

However, the primary drawback of DDPMs is their inference speed. Standard models begin with pure Gaussian noise and require hundreds or thousands of iterative denoising steps to produce a clean segmentation mask. This high computational cost hinders their adoption in real-time clinical workflows. While existing acceleration techniques like distillation or latent space mapping exist, they often compromise image quality or introduce training instabilities.

Methodology

The researchers propose PD-DDPM (Pre-segmentation Diffusion Sampling), a "warm start" strategy designed to bypass the most computationally expensive parts of the diffusion process. Instead of starting from total chaos (pure noise), the pipeline follows three stages:

  1. Pre-segmentation: A standard, separately trained network (e.g., Attention U-Net) generates an initial "rough draft" segmentation ($x_{pre}$).
  2. Forward Diffusion Shortcut: This draft is corrupted with a specific amount of noise to reach an intermediate state ($T'$) rather than the full noise state ($T$).
  3. Reverse Denoising: The diffusion model begins refinement from this noisy intermediate state, requiring significantly fewer steps to reach the final segmentation ($x_0$).
Comparison of the standard Vanilla DDPM (left) and the proposed PD-DDPM (right) architectures, highlighting the shortcut from pre-segmentation.

This approach was validated using the WMH (White Matter Hyperintensities) dataset from the MICCAI 2017 challenge, consisting of 60 brain MRI cases. The task involved segmenting lesions associated with neurodegenerative diseases using 2D multi-slice FLAIR and 3D T1-weighted images.

Key Insights and Results

The core innovation of PD-DDPM is transforming the diffusion model from a "creator" into a "refiner." By leveraging a pre-segmentation prior, the model focuses on correcting initial errors rather than generating a mask from scratch.

Performance and Efficiency

  • Accuracy: PD-DDPM achieved a Dice score of 0.812, surpassing both the vanilla DDPM (0.796) and the standalone Attention U-Net (0.799).
  • Boundary Precision: The 95th percentile Hausdorff Distance (HD95) improved to 3.494, down from 4.179 in the vanilla model.
  • Speed: The model achieved these results in just 300 steps, representing a 3.3x speedup over the standard 1000-step process.

Uncertainty and Ensembling

A major advantage of this framework is the ability to generate uncertainty maps through stochastic sampling. By running the 300-step refinement multiple times, an implicit ensemble is created. Averaging just five masks was found to significantly boost the Dice score while reducing variance.

Qualitative results showing Flair images, ground truth masks, PD-DDPM segmentations, and corresponding uncertainty maps.

The "Goldilocks" Zone

The researchers identified a critical trade-off regarding the starting point $T'$. If $T'$ is too low, the model cannot fix the pre-segmenter's errors; if it is too high, the speed benefits and the useful prior information are lost. $T'=300$ was found to be the optimal balance for this task.

Conclusion and Future Work

PD-DDPM demonstrates that diffusion models can be both accurate and efficient for medical segmentation by incorporating architectural priors. While the method requires training two components and careful hyperparameter tuning for $T'$, its orthogonality allows it to be combined with any state-of-the-art segmentation network. Future research will focus on extending this "warm start" approach to full 3D volumes to better capture spatial context in medical scans.

Summary updated

Abstract

Based on the Denoising Diffusion Probabilistic Model (DDPM), medical image segmentation can be described as a conditional image generation task, which allows to compute pixel-wise uncertainty maps of the segmentation and allows an implicit ensemble of segmentations to boost the segmentation performance. However, DDPM requires many iterative denoising steps to generate segmentations from Gaussian noise, resulting in extremely inefficient inference. To mitigate the issue, we propose a principled acceleration strategy, called pre-segmentation diffusion sampling DDPM (PD-DDPM), which is specially used for medical image segmentation. The key idea is to obtain pre-segmentation results based on a separately trained segmentation network, and construct noise predictions (non-Gaussian distribution) according to the forward diffusion rule. We can then start with noisy predictions and use fewer reverse steps to generate segmentation results. Experiments show that PD-DDPM yields better segmentation results over representative baseline methods even if the number of reverse steps is significantly reduced. Moreover, PD-DDPM is orthogonal to existing advanced segmentation models, which can be combined to further improve the segmentation performance.
BibTeX
@inproceedings{guo2023accelerating,
  author = {Xutao Guo and Yanwu Yang and Chenfei Ye and Shang Lu and Yang Xiang and Ting Ma},
  title = {{Accelerating Diffusion Models via Pre-segmentation Diffusion Sampling for Medical Image Segmentation}},
  booktitle = {ISBI},
  year = {2023},
  doi = {10.1109/isbi53787.2023.10230524},
}
Figure