Accelerating Diffusion Models Via Pre-Segmentation Diffusion Sampling for Medical Image Segmentation

Xutao Guo, Yanwu Yang, Chenfei Ye, Shang Lu, Bo Peng, Hua Huang, Yang Xiang, Ting Ma

TL;DR

Researchers have developed PD-DDPM (Pre-segmentation Diffusion Sampling), a method that significantly speeds up medical image segmentation using diffusion models. By starting the denoising process from a noisy "rough draft" provided by a standard neural network—rather than starting from pure random noise—the model achieves higher accuracy (Dice score of 0.812 vs. 0.796) with 70% fewer computational steps.

What Problem Does This Paper Address?

Denoising Diffusion Probabilistic Models (DDPMs) have taken the AI world by storm, but they have a major "speed limit." In medical imaging, DDPMs are highly valued because they are probabilistic: instead of giving one "best guess" for a tumor or lesion boundary, they can generate multiple hypotheses. This allows doctors to see uncertainty maps, highlighting areas where the AI is unsure—a critical feature for avoiding misdiagnosis.

The catch? DDPMs are slow. They typically start with a block of pure Gaussian noise and iteratively refine it over hundreds or thousands of steps to produce a clean segmentation mask. This high "inference cost" makes them difficult to use in real-time clinical settings. While other researchers have tried to accelerate diffusion via distillation or latent space mapping, these methods often sacrifice image quality or require complex, unstable training (like GANs).

Methods

The authors propose a "warm start" strategy called PD-DDPM. Instead of the standard two-step process (Forward Diffusion $\to$ Reverse Denoising from pure noise), they introduce a three-stage pipeline:

  1. Pre-segmentation: A standard, separately trained segmentation network (like an Attention U-Net) takes the medical image and produces a "pre-segmentation" result ($x_{pre}$).
  2. Forward Diffusion (The Shortcut): This pre-segmentation isn't perfect, so the authors "corrupt" it by adding a specific amount of noise according to the standard diffusion rule. This brings the image to an intermediate noisy state ($T'$) rather than the total-chaos state ($T$).
  3. Reverse Denoising: The diffusion model starts its work from this noisy intermediate state ($T'$) and performs far fewer steps to reach the final, refined segmentation ($x_0$).

Crucially, this method is orthogonal to other improvements; you can swap in any advanced segmentation network as the "pre-segmenter" to further boost performance.

Datasets

The model was evaluated on the WMH (White Matter Hyperintensities) dataset from the MICCAI 2017 challenge.

  • Content: 60 cases of brain MRI images (3D T1-weighted and 2D multi-slice FLAIR).
  • Task: Segmenting white matter hyperintensities (often associated with aging and neurodegenerative diseases).
  • Setup: The data was split into 5 folds for robust cross-validation, with images cropped to $128 \times 192$ patches.

Key Insights and Ideas

  • The "Aha" Moment: The authors realized that starting from pure noise is inefficient for segmentation because we often already have "decent" segmenters. By using a pre-segmentation as a prior, the diffusion model acts more like a refiner than a creator, correcting the errors of the initial network.
  • Optimal "Noise" Level: There is a sweet spot for the starting point ($T'$). If $T'$ is too small, the model can't fix the pre-segmenter's mistakes. If $T'$ is too large, you lose the speed benefits and the "hint" provided by the pre-segmentation. They found $T'=300$ (out of 1000) to be the "Goldilocks" zone.
  • Implicit Ensembling: Because the process is still stochastic, you can run the 300-step refinement multiple times to create an ensemble. The authors found that averaging just 5 masks significantly boosted the Dice score and reduced variance.

Key Results

  • Superior Accuracy: PD-DDPM achieved a Dice score of 0.812, outperforming the vanilla DDPM (0.796) and the standalone Attention U-Net (0.799).
  • Better Boundaries: The 95th percentile Hausdorff Distance (HD95), which measures boundary errors, dropped to 3.494, compared to 4.179 for the vanilla diffusion model.
  • Efficiency: The model achieved these top-tier results using only 300 steps, a 3.3x speedup over the standard 1000-step approach.
  • Robustness: Even when the "pre-segmenter" was relatively weak (Dice of 0.754), the PD-DDPM was able to "rescue" the result and pull it up to 0.790.

Limitations and Future Work

  • Training Overhead: While inference is faster, the method requires training two separate components: the pre-segmentation network and the diffusion model.
  • Hyperparameter Sensitivity: The optimal $T'$ might vary depending on the complexity of the medical task (e.g., segmenting a large organ vs. tiny lesions).
  • 2D vs 3D: This study focused on 2D slices. Extending this "warm start" diffusion to full 3D volumes is a logical next step but will require significantly more memory and compute.

Summary updated

Abstract

Based on the Denoising Diffusion Probabilistic Model (DDPM), medical image segmentation can be described as a conditional image generation task, which allows to compute pixel-wise uncertainty maps of the segmentation and allows an implicit ensemble of segmentations to boost the segmentation performance. However, DDPM requires many iterative denoising steps to generate segmentations from Gaussian noise, resulting in extremely inefficient inference. To mitigate the issue, we propose a principled acceleration strategy, called pre-segmentation diffusion sampling DDPM (PD-DDPM), which is specially used for medical image segmentation. The key idea is to obtain pre-segmentation results based on a separately trained segmentation network, and construct noise predictions (non-Gaussian distribution) according to the forward diffusion rule. We can then start with noisy predictions and use fewer reverse steps to generate segmentation results. Experiments show that PD-DDPM yields better segmentation results over representative baseline methods even if the number of reverse steps is significantly reduced. Moreover, PD-DDPM is orthogonal to existing advanced segmentation models, which can be combined to further improve the segmentation performance.
BibTeX
@inproceedings{guo2023accelerating,
  author = {Xutao Guo and Yanwu Yang and Chenfei Ye and Shang Lu and Bo Peng and Hua Huang and Yang Xiang and Ting Ma},
  title = {{Accelerating Diffusion Models Via Pre-Segmentation Diffusion Sampling for Medical Image Segmentation}},
  booktitle = {ISBI},
  year = {2023},
  doi = {10.1109/isbi53787.2023.10230524},
}

Loading related content...

Figure