Introduction
Diffusion models have revolutionized image synthesis, achieving state-of-the-art results in visual fidelity. However, their primary drawback remains the high computational cost: generating a single image typically requires hundreds of iterative denoising steps. While existing schedulers like DDIM or DPM-Solver attempt to accelerate this process by skipping steps, they often suffer from significant quality degradation when pushed to extremely low step counts (e.g., fewer than 10 steps).
This paper introduces Optimal Linear Subspace Search (OLSS), a data-driven approach to scheduler design. By treating the generation process as an expansion within a linear subspace, OLSS enables high-quality image synthesis in as few as 5 steps—achieving sub-second inference on modern hardware without requiring expensive model retraining or distillation.
The Challenge of Iterative Denoising
Traditional acceleration methods generally fall into two categories: mathematical ODE/SDE solvers and model distillation. Mathematical solvers approximate the denoising path using general numerical theories, which can become inaccurate at high speeds. Distillation involves training a new, smaller model to mimic the original, a process that is both computationally intensive and difficult to scale.
OLSS seeks a middle ground: a lightweight, training-free scheduler that "learns" the optimal shortcut through the generation process by observing the specific behavior of a pre-trained model.
Methodology: Optimal Linear Subspace Search
The core innovation of OLSS lies in its shift from fixed mathematical formulas to a data-driven coefficient learning strategy.
1. Linear Subspace Analysis
The authors demonstrate that the generation process can be accurately modeled as the expansion of a linear subspace spanned by the initial noise and previous model outputs. Analysis of intermediate variables reveals high correlation and redundancy, suggesting that the complex, non-linear path of a diffusion model can be approximated within a lower-dimensional linear space.
2. Coefficient Learning and Path Optimization
Instead of relying on hard-coded calculus coefficients, OLSS uses a simple linear model to predict the next state. The process involves:
- Calibration: Running a full 1000-step generation on a small sample (e.g., 32 images) to collect ground-truth data.
- Least Squares Optimization: Solving for coefficients that minimize approximation error for a target step count.
- Binary Search Pathfinding: Identifying the optimal sequence of time steps (the "path") to minimize cumulative error, particularly in early, high-risk stages of generation.
Experimental Evaluation
The researchers evaluated OLSS using Stable Diffusion (v1.4 and v2.1) across open-domain prompts and fine-tuned datasets like CelebA-HQ and LSUN-Church.
Key Results
- High-Speed Quality: At 5 and 10 steps, OLSS consistently outperformed standard schedulers like DDIM and DPM-Solver++ in terms of Frechet Inception Distance (FID).
- Efficiency: The method achieves high-quality results in under one second on an NVIDIA A100 GPU.
- Visual Fidelity: Unlike traditional solvers that may produce blurry results at low step counts, OLSS preserves fine textures and details, making 5-step outputs nearly indistinguishable from 1000-step benchmarks.
Conclusion
Optimal Linear Subspace Search (OLSS) demonstrates that the "shortcut" to fast diffusion is linear. By leveraging a small amount of calibration data, OLSS finds more efficient denoising paths than general-purpose mathematical solvers. While it requires a one-time calibration step, its ability to provide high-fidelity images at extreme speeds without model retraining makes it a powerful tool for deploying diffusion models in real-time applications.
Summary updated