Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation

Yuli Wu, Weidong He, Dennis Eschweiler, Ningxin Dou, Zixin Fan, Shengli Mi, Peter Walter, Johannes Stegmaier

TL;DR

Researchers have developed a method to synthesize realistic retinal Optical Coherence Tomography (OCT) images using Denoising Diffusion Probabilistic Models (DDPMs). By transforming rough hand-drawn-style sketches into high-fidelity scans and using a "teacher" model to generate accurate labels, they significantly boosted the performance of automated layer segmentation, even proving that models trained entirely on synthetic data can rival those trained on real human-annotated images.

What Problem Does This Paper Address?

In the world of medical AI, data is gold, but annotated data is a diamond—rare and expensive to produce. For retinal OCT (a non-invasive imaging technique used to diagnose diseases like glaucoma), training deep learning models to segment delicate retinal layers requires thousands of precisely labeled images.

The challenge is twofold:

  1. Scarcity: There aren't enough expert-annotated datasets.
  2. Alignment: While Generative Adversarial Networks (GANs) can make fake images, they often struggle with structural consistency, making it hard to ensure the "ground truth" labels actually match the synthesized anatomy.

Methods

The authors propose a pipeline based on Denoising Diffusion Probabilistic Models (DDPMs). Unlike standard diffusion which starts from pure white noise, this method uses a "short-cut" approach:

  1. Sketch Generation: They create "sketches" of retinal layers based on the statistical distribution of layer thicknesses and intensities found in a small real dataset.
  2. Guided Synthesis: They feed these sketches into the DDPM. Instead of starting the denoising process from scratch ($t=T$), they start at an intermediate timestep ($t_{start}=300$). This allows the model to preserve the global structure of the sketch while "filling in" realistic textures, noise, and biological details.
  3. Knowledge Adaptation (The Teacher-Student Trick): The authors noticed that synthesized images don't always perfectly align with the original input sketches (e.g., a layer boundary might shift slightly during denoising). To fix this, they used a "Teacher" model (a $U^2$-Net trained on real data) to generate new, highly accurate "pseudo-labels" for the synthetic images. These pairs are then used to train "Student" models.

Datasets

The study utilized the GOALS Challenge dataset (MICCAI 2022), which consists of 100 circumpapillary OCT images.

  • train-50: 50 images used for training the DDPM and the Teacher model.
  • test-50: 50 images reserved for final evaluation.
  • Layers targeted: Retinal Nerve Fiber Layer (RNFL), Ganglion Cell-Inner Plexiform Layer (GCIPL), and the Choroid Layer (CL).

Key Insights and Ideas

  • The "Sketch-to-Photo" Sweet Spot: The researchers found that $t_{start}=300$ (out of 400 steps) was the "Goldilocks" zone. Any lower, and the images looked too much like the fake sketches; any higher, and the model ignored the sketch entirely, creating random (though realistic) anatomy.
  • Labels are Fluid: One of the most important realizations was that the input sketch shouldn't be treated as the final ground truth. Because the DDPM introduces its own realistic "histological structures," the labels must be "distilled" back from the synthetic image using a pre-trained teacher model to ensure pixel-perfect alignment.
  • Synthetic-Only Training: The paper demonstrates that a model trained on 1,000 synthetic images (and zero real ones) can achieve a Dice score (a measure of overlap) comparable to a model trained on 50 real, expert-annotated images.

Key Results

The researchers tested their synthetic data across five different architectures (U-Net, $U^2$-Net, FCN-ResNet, DeepLabv3+, and TransUNet).

  • Performance Boost: Adding 1,000 synthetic images to the 50 real images improved the average Dice score significantly. For example, the standard U-Net's score jumped from 84.70% to 91.25%.
  • Choroid Layer Improvement: The Choroid layer is notoriously difficult to segment because its lower boundary is often faint. The synthetic data helped models learn these subtle boundaries much more effectively.
  • Robustness: The improvement was consistent across all five neural network architectures, proving the synthetic data is "generalizable" and not just overfitted to one specific model type.

Limitations and Future Work

  • Statistical Bias: The sketches are currently generated based on the statistics of the train-50 dataset. This means the synthetic images mostly reflect "normal" or specific anatomy seen in that small sample.
  • Pathology: The current work focuses on healthy or standard structures. The authors suggest that future work should involve facilitating "pathological intervention"—essentially teaching the DDPM to synthesize specific retinal diseases or lesions.
  • Domain Adaptation: The authors see potential in using this for unsupervised domain adaptation, helping models trained on one OCT scanner work effectively on images from a different manufacturer's machine.

Summary updated

Abstract

Modern biomedical image analysis using deep learning often encounters the challenge of limited annotated data. To overcome this issue, deep generative models can be employed to synthesize realistic biomedical images. In this regard, we propose an image synthesis method that utilizes denoising diffusion probabilistic models (DDPMs) to automatically generate retinal optical coherence tomography (OCT) images. By providing rough layer sketches, the trained DDPMs can generate realistic circumpapillary OCT images. We further find that more accurate pseudo labels can be obtained through knowledge adaptation, which greatly benefits the segmentation task. Through this, we observe a consistent improvement in layer segmentation accuracy, which is validated using various neural networks. Furthermore, we have discovered that a layer segmentation model trained solely with synthesized images can achieve comparable results to a model trained exclusively with real images. These findings demonstrate the promising potential of DDPMs in reducing the need for manual annotations of retinal OCT images.
BibTeX
@inproceedings{wu2024retinal,
  author = {Yuli Wu and Weidong He and Dennis Eschweiler and Ningxin Dou and Zixin Fan and Shengli Mi and Peter Walter and Johannes Stegmaier},
  title = {{Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation}},
  booktitle = {ISBI},
  year = {2024},
  doi = {10.1109/isbi56570.2024.10635836},
}

Loading related content...

Figure