Leapfrog Latent Consistency Model (LLCM) for Medical Images Generation

Lakshmikar R. Polamreddy, Kingshuk Roy, Sheng‐Han Yueh, Youshan Zhang, Shilpa Kuppili, Jialu Li, Deepshikha Mahato2024

Introduction

The development of robust AI models for medical imaging is frequently hindered by the "data desert"—a scarcity of high-quality, diverse, and accessible medical datasets. Unlike general computer vision, which benefits from massive repositories like ImageNet, medical data is often restricted by privacy regulations such as HIPAA and GDPR. Furthermore, state-of-the-art generative models, such as Diffusion Models, are computationally intensive, requiring numerous iterative steps to produce high-fidelity outputs. This latency limits their utility in real-time clinical applications and large-scale synthetic data augmentation.

To address these challenges, researchers have introduced the Leapfrog Latent Consistency Model (LLCM). This framework is designed to generate high-resolution (512x512) medical images in as few as 1 to 4 steps, significantly outpacing traditional diffusion models while maintaining clinical accuracy.

Methodology

The LLCM architecture is built upon a pre-trained diffusion model, optimized through a process known as consistency distillation.

The Leapfrog Solver and PF-ODE

The core innovation of LLCM lies in its treatment of the reverse diffusion process as a Probability Flow Ordinary Differential Equation (PF-ODE). Traditional solvers navigate this flow using small, incremental steps, which leads to high inference times. In contrast, the LLCM employs a Leapfrog algorithm within the latent space. This numerical method allows the model to "jump" through time steps, enabling rapid convergence to a high-quality image.

Consistency Distillation and Guidance

By enforcing "self-consistency"—ensuring that different points along the same flow trajectory map back to the same initial image—the model learns to predict the final output in a single leap or a minimal number of steps (1-4). Additionally, Classifier-Free Guidance (CFG) is integrated to ensure that generated images remain strictly aligned with text prompts, such as specific anatomical structures or pathological conditions.

The MedImgs Dataset

A critical component of this research is the introduction of MedImgs, a comprehensive benchmark dataset for medical image generation.

  • Scale and Diversity: The dataset comprises 250,127 images covering 61 disease types and 159 distinct classes.
  • Cross-Species Scope: Uniquely, MedImgs includes both human and animal data (e.g., cattle, dogs, fish), spanning various modalities and body parts.
  • Standardization: Data was aggregated from open-source platforms like GitHub and Kaggle, then cleaned and standardized to the JPG format.
Table 1: Human body parts and associated disease categories included in the MedImgs dataset.

The inclusion of animal data suggests a "One Health" approach, where features learned from veterinary medicine can enhance the robustness of human medical AI models.

Key Results and Discussion

The LLCM demonstrates superior performance in both speed and image quality compared to established benchmarks.

Step Efficiency and Visual Fidelity

A significant finding is that medical image quality does not scale linearly with the number of inference steps. The LLCM achieved its peak performance—measured by the lowest Fréchet Inception Distance (FID)—at just 4 steps. At this stage, LLCM recorded an FID of 145.68, vastly outperforming Stable Diffusion (249.18) and Dreambooth (300.15) at the same step count.

Figure 1: Visual comparison of generated medical images across different inference steps (1, 2, 4, 8, 20) for various modalities including MRI, Histology, and CT.

As shown in the figure above, the model produces clear, anatomically coherent structures even at very low step counts. By performing integration in the compressed latent space rather than raw pixel space, the model maintains high resolution without excessive memory overhead.

Generalization

The model's robustness was further validated on an unseen dataset of dog cardiac X-rays. LLCM produced clearer and more anatomically correct results than Stable Diffusion or DDPM, which often yielded blurry or noisy outputs on the same task.

Limitations and Future Directions

Despite its success, the LLCM exhibits a performance imbalance; image quality for human diseases remains higher than for animal diseases, likely due to the higher prevalence of human data in the training set. Future work will focus on:

  1. One-Step Generation: Refining the distillation process to achieve peak quality in a single inference pass.
  2. Dataset Expansion: Incorporating more rare diseases and 3D modalities such as MRI and CT volumes.
  3. Cross-Species Transfer: Further exploring how veterinary data can improve human diagnostic models.

The LLCM represents a significant step forward in making high-fidelity medical image generation accessible and efficient, providing a powerful tool for data augmentation in the medical AI field.

Summary updated

Abstract

The scarcity of accessible medical image data poses a significant obstacle in effectively training deep learning models for medical diagnosis, as hospitals refrain from sharing their data due to privacy concerns. In response, we gathered a diverse dataset named Med-Imgs, which comprises over 250,127 images spanning 61 disease types and 159 classes of both humans and animals from open-source repositories. We propose a Leapfrog Latent Consistency Model (LLCM), a large vision model for the generation of medical images. It is distilled from a retrained diffusion model based on the collected MedImgs dataset, which enables our model to generate real-time high-resolution images. We formulate the reverse diffusion process as a probability flow ordinary differential equation (PF-ODE) and solve it in latent space using the Leapfrog algorithm. This formulation enables rapid sampling without necessitating additional iterations. Our model demonstrates state-of-the-art performance in generating medical images. Furthermore, our model can be fine-tuned with any custom medical image datasets, facilitating the generation of a vast array of images. Our experimental results outperform those of existing models on unseen dog cardiac X-ray images. Source code is available at https://github.com/LakshmikarPolamreddy/LLCM.
BibTeX
@misc{polamreddy2024leapfrog,
  author = {Lakshmikar R. Polamreddy and Kingshuk Roy and Sheng‐Han Yueh and Youshan Zhang and Shilpa Kuppili and Jialu Li and Deepshikha Mahato},
  title = {{Leapfrog Latent Consistency Model (LLCM) for Medical Images Generation}},
  year = {2024},
  eprint = {2411.15084},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.15084},
}
Figure