Abstract
Medical imaging datasets typically do not contain many training images, usually being deficient for training deep learning networks. We propose a deep residual variational auto-encoder and a generative adversarial network that can generate a synthetic retinal fundus image dataset with corresponding blood vessel annotation. Our initial experiments produce results with higher scores than the state-of-the-art for verifying that the structural statistics of our generated images are compatible with real fundus images. The successful application of generative models to generate synthetic medical data will not only help to mitigate the small dataset problem but will also address the privacy concerns associated with medical datasets.
Introduction
Deep learning thrives on massive datasets, but medical imaging is notoriously "data-starved." High-quality, annotated datasets like retinal fundus images—used to diagnose conditions like glaucoma or diabetic retinopathy—are difficult to obtain due to patient privacy concerns and the high cost of expert annotation.
Existing synthetic methods often struggle with two primary challenges: low resolution (often limited to 256x256) and poor structural integrity, where generated vessel networks lack biological plausibility. This research introduces a hybrid generative pipeline to create high-resolution (up to 1024x1024) synthetic retinal fundus images along with their corresponding blood vessel maps, requiring as few as 20 training images.
Methodology
The proposed multi-stage pipeline moves from abstract noise to a high-fidelity medical image through several specialized components:
- Deep Residual VAE: A Variational Auto-encoder with residual layers generates initial blood vessel annotations. The residual layers are critical for preserving fine structural details that standard VAEs often blur due to $L_2$ loss.
- De-noising: A deconvolution-based algorithm sharpens the VAE's output vessel maps.
- Image-to-Image Translation (Pix2pix): A GAN architecture "paints" realistic retinal textures over the synthetic vessel maps, ensuring anatomical alignment between the "map" and the final fundus image.
- Super-Resolution (EDSR): An Enhanced Deep Super-Resolution network boosts the final paired output from 512x512 to a crisp 1024x1024 resolution.
Results and Comparison
The model was trained using only 20 images from the DRIVE (Digital Retinal Images for Vessel Extraction) dataset, supplemented by data augmentation. Despite this remarkably small training set, the pipeline achieves state-of-the-art performance in structural realism and resolution.
As shown in the comparison table below, the proposed method is the only one to offer both blood vessel annotations and high-resolution outputs (1024x1024) while maintaining high data efficiency. Quantitatively, the researchers used the Structural Similarity Index Measure (SSIM) to evaluate realism; their model achieved an SSIM of 0.74, significantly outperforming the previous benchmark of 0.64.
Discussion and Conclusion
By decoupling structure (vessel map) and texture (fundus image), the model avoids common GAN pitfalls where realistic textures lack anatomical sense. The success of this "small data" approach suggests that generative models can effectively learn the inherent distribution of complex medical structures from very few examples.
While the results are promising, future work should include clinical validation by ophthalmologists to test the indistinguishability of synthetic images from real ones. Additionally, the authors suggest this pipeline could be adapted for other medical imaging tasks, such as MRI or CT scan synthesis, to further address data scarcity and privacy concerns in the field.
Summary updated