Diffusion models generate high-fidelity data by reversing a gradual noisy process. This generation path is typically modeled as a trajectory along a Probability Flow Ordinary Differential Equation (ODE). To produce a single sample, a standard diffusion model must evaluate a large neural network dozens or hundreds of times, step by step. This iterative requirement creates a massive computational bottleneck, rendering real-time deployment challenging across various applications.
Halving the Trajectory: Progressive Distillation
To bypass this bottleneck, researchers developed progressive distillation. The core intuition is simple: if a "teacher" model requires two sequential steps to move from a noisy state $x_t$ to a cleaner state $x_{t-2}$ via an intermediate state $x_{t-1}$, we can train a "student" model to predict $x_{t-2}$ directly from $x_t$ in a single step.
This concept was formalized by Tim Salimans et al. in Progressive Distillation for Fast Sampling of Diffusion Models. By recursively applying this step-halving procedure—where the student from one round becomes the teacher for the next—the total number of sampling steps decreases logarithmically (e.g., from 1024 to 512, then 256, down to 4 or 2 steps) while preserving the generative quality of the original model.
This trajectory compression framework has been widely adapted. For instance, Chenlin Meng et al. in On Distillation of Guided Diffusion Models extended progressive distillation to guided models, collapsing the computationally expensive classifier-free guidance (CFG) loop into a fast, few-step student network. Similarly, Yi-Ting Hsiao et al. in Plug-and-Play Diffusion Distillation utilized progressive distillation to compress sampling trajectories by substituting heavy guidance mechanisms with lightweight external models.
Enforcing Global Trajectory Consistency
While progressive distillation relies on an iterative, multi-stage halving process, an alternative approach focuses on learning a global mapping function known as consistency distillation.
As explained by Danush Kumar Venkatesh et al. in SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Models, consistency models train the network to map any point along a given Probability Flow ODE trajectory directly back to its origin ($t=0$). If every intermediate point on the path is forced to predict the exact same endpoint, the model can theoretically generate a clean sample in a single evaluation, or a handful of steps to refine details.
This approach eliminates the need for multiple sequential training phases. The authors of several recent works have leveraged this property to achieve massive speedups. For example, Yeongtak Oh et al. in Efficient Diffusion-Driven Corruption Editor for Test-Time Adaptation applied consistency distillation to achieve near real-time image restoration, making generative models competitive with fast, non-diffusion baselines.
Direct Jumps vs. Sequential Halving
A key trade-off in trajectory compression is how the student learns to approximate the teacher's path. Progressive distillation can suffer from an accumulation of errors across successive halving generations. To mitigate this, some frameworks opt for direct multi-step distillation.
Pavlo Chemerys et al. demonstrate this in SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds, where the student is trained to match the teacher's multi-step output in a single large jump rather than through a sequence of intermediate stages. This direct mapping prevents the compounding errors typical of multi-phase progressive distillation, enabling high-fidelity text-to-image generation on mobile hardware in under two seconds.
Cross-Modal and Cross-Domain Versatility
The principle of compressing diffusion trajectories is not limited to standard image generation; it has fundamentally transformed efficiency across diverse domains:
- Speech and Audio Synthesis: In text-to-speech (TTS) systems, Rongjie Huang et al. developed ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech to reduce sampling iterations down to two steps. Similarly, Tasnima Sadekova et al. in Fast Grad-TTS: Towards Efficient Diffusion-Based Speech Generation on CPU demonstrated that collapsing the denoising process allows high-quality speech synthesis to run in real-time on consumer-grade CPUs.
- 3D Avatars and Video: Generating consistent video or 3D structures frame-by-frame is notoriously slow due to the repeated evaluations required per frame. Haijie Yang in ConsistentAvatar: Learning to Diffuse Fully Consistent Talking Head Avatar with Temporal Guidance integrated latent consistency models to cut inference steps from 1,000 to 10. For speech-driven facial animations, Peng Chen et al. in DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation and Jiaqi Yang et al. in FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation compressed the generation path down to a minimal number of steps (as few as six) to maintain strict temporal and anatomical consistency.
- Inpainting and Editing: Ruibin Li et al. in RORem: Training a Robust Object Remover with Human-in-the-Loop combined latent consistency modeling with low-rank adaptation to enable sub-second generative object removal.
- Combinatorial Optimization: Beyond perceptual data, Junwei Huang et al. in Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation applied progressive distillation to NP-complete problems like the Traveling Salesman Problem, achieving a 16x speedup with virtually no loss in solution quality.
- Language Modeling: Moving past continuous spaces, Justin Deschenaux et al. in Beyond Autoregression: Fast LLMs via Self-Distillation Through Time introduced a progressive distillation schedule for discrete diffusion language models, allowing text generation to break the traditional token-by-token autoregressive bottleneck.
By framing the problem as trajectory compression—whether through step-halving, direct jumps, or consistency mappings—distillation bridges the gap between the high-quality generative capabilities of diffusion models and the strict latency requirements of real-world deployment.
Key Foundational Work
On Distillation of Guided Diffusion Models (2022) — contributed 2 concept elements to this direction.
Fast Grad-TTS: Towards Efficient Diffusion-Based Speech Generation on CPU (2022) — contributed 2 concept elements to this direction.
Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation (2023) — contributed 1 concept elements to this direction.