Denoising diffusion models have revolutionized generative AI, producing highly detailed images, videos, and structural predictions. However, their exceptional quality comes at a steep price: inference latency. To generate a single sample, standard diffusion models must iteratively reverse a degradation process over dozens, hundreds, or even thousands of sequential steps.
This iterative sampling bottleneck stems from the underlying mathematics of diffusion. Generating an image requires solving an Ordinary Differential Equation (ODE) or Stochastic Differential Equation (SDE) across a complex probability trajectory. If a solver takes steps that are too large, it drifts off the true data manifold, resulting in severe artifacts or blurry outputs. Consequently, real-time deployment remains a significant challenge across applications ranging from text-to-image generation to medical imaging, as noted by Chenlin Meng et al. in their work on guided diffusion models, and by Julia Wolleb et al. in medical anomaly detection.
To bypass this bottleneck, a major research direction focuses on compressing or eliminating these iterative steps entirely, collapsing the multi-step denoising process into few-step or even single-step generation.
Straightening the Trajectory via Flow Matching
To understand why traditional diffusion models require so many steps, imagine navigating a highly winding mountain road. To stay on the road (the data manifold), a driver must make frequent, tiny adjustments. If the road were perfectly straight, however, the driver could reach the destination in a single, linear acceleration.
This is the core intuition behind trajectory straightening techniques like Rectified Flow. In standard diffusion, the probability paths between random noise and clean data are highly curved due to the stochastic nature of the forward noise schedule. Rectified flows re-formulate the transport process to make the trajectories between noise and images as straight as possible.
By straightening these paths, the model minimizes discretization errors when taking large steps. As demonstrated in InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation, straightening the probability flows allows a complex, multi-step trajectory to be distilled into a single, ultra-fast inference step taking less than a tenth of a second without sacrificing structural coherence.
Knowledge Distillation and Distribution Matching
Another prominent strategy is knowledge distillation, where a fast "student" network is trained to mimic a slow, high-quality "teacher" diffusion model.
Crucially, simple pixel-level regression does not work for one-step distillation. If a student model is trained to minimize the mean squared error between its single-step output and the teacher's multi-step output, it encounters the "regression to the mean" problem. Because a single noise vector could plausibly resolve into many different clean images, pixel-level minimization forces the student to average all possible outcomes, yielding blurry results.
To overcome this, advanced distillation methods match high-level features or entire probability distributions instead of raw pixels:
- Feature Matching: Defang Chen et al. introduce Classifier-based Feature Distillation (CFD), which forces the student to match the high-level semantic features of the teacher. This allows the student to capture structural and contextual meaning, enabling high-fidelity generation in 1 to 4 steps.
- Score and Density Matching: Rather than comparing outputs directly, methods like Score Implicit Matching (SIM), developed by Weijian Luo et al., perform data-free distillation by aligning the implicit score functions of the student and teacher. Similarly, Yuanzhi Zhu et al. propose Di-Bregman, a framework that uses Bregman density-ratio matching to systematically drive the divergence between the student and teacher distributions to zero.
- Guidance Alignment: For data-free text-to-image generation, Hai Huang demonstrates that a "Long-Short Guidance" (SiD-LSG) strategy can successfully bridge the performance gap by utilizing specialized guidance signals to align the student's single-step outputs with the teacher's target distribution.
Hybridizing Diffusion with Adversarial Objectives
Generative Adversarial Networks (GANs) are inherently single-pass architectures; they map noise to data in one forward pass because they rely on a discriminator to enforce realism. By hybridizing diffusion models with adversarial objectives, researchers can leverage the structural stability of diffusion training while inheriting the single-step speed of GANs.
In this paradigm, the diffusion model acts as the generator, and an auxiliary discriminator evaluates the realism of the generated outputs. This eliminates the need for intermediate denoising steps because the discriminator penalizes the blurriness and artifacts typically caused by large step sizes.
For instance, yanwu xu et al. developed UFOGen, a diffusion-GAN hybrid that utilizes pre-trained Stable Diffusion weights but modifies the objective function to incorporate an adversarial loss, achieving large-scale text-to-image synthesis in a single step. To prevent the adversarial training from collapsing or producing unnatural artifacts, Dar-Yen Chen introduces NitroFusion, which employs a dynamic, constantly refreshing pool of specialized discriminators to act as a robust panel of "art critics." For domain-specific tasks like face restoration, Jingkai Wang et al. combine this adversarial guidance with a visual representation embedder in OSDFace to achieve identity-consistent, single-step restorations.
Bypassing Guidance and Solver Bottlenecks
Beyond reducing the number of steps, researchers are also targeting the computational overhead within each individual step.
A major culprit behind per-step latency is Classifier-Free Guidance (CFG). To improve text alignment, standard diffusion models evaluate the network twice per step—once conditioned on the text prompt and once unconditioned. This effectively doubles the inference time.
To resolve this, Yara Bahram et al. developed DogFit, a domain-guided fine-tuning method that internalizes the guidance signal directly into the model's weights during training, allowing for highly controlled generation with only a single forward pass per step. In an analysis of distribution matching distillation, Dongyang Liu et al. decoupled the roles of guidance and distribution matching, revealing that CFG acts primarily as an engine for few-step generation and can be optimized independently via distinct noise schedules.
Alternatively, the sampling bottleneck can be mitigated without retraining the underlying model by optimizing the mathematical solvers themselves. Soochul Park introduces Dual-Solver, a learned ODE solver that dynamically interpolates between different prediction types (such as noise, data, or velocity) to maintain high image quality in ultra-low step regimes (3 to 9 steps). Taking a discrete approach, Ping Chen et al. frame the denoising sequence as a shortest-path graph problem in ShortDF, optimizing residual propagation to minimize reconstruction errors in as few as 2 steps.
Key Foundational Work
Accelerating Diffusion Sampling with Classifier-based Feature Distillation (2022) — contributed 1 concept elements to this direction.
On Distillation of Guided Diffusion Models (2022) — contributed 1 concept elements to this direction.
Diffusion Models for Medical Anomaly Detection (2022) — contributed 1 concept elements to this direction.