Abstract
Diffusion Probabilistic Models (DPMs) have demonstrated exceptional performance in generative tasks, but this comes at the expense of sampling efficiency. To enhance sampling speed without sacrificing quality, various distillation-based accelerated sampling algorithms have been recently proposed. However, they typically require significant additional training costs and model parameter storage, which limit their practical application. In this work, we propose PCA-based Adaptive Search (PAS), which optimizes existing solvers for DPMs with minimal learnable parameters and training costs. Specifically, we first employ PCA to obtain a few orthogonal unit basis vectors to span the high-dimensional sampling space, which enables us to learn just a set of coordinates to correct the sampling direction; furthermore, based on the observation that the cumulative truncation error exhibits an "S"-shape, we design an adaptive search strategy that further enhances the sampling efficiency and reduces the number of stored parameters to approximately 10. Extensive experiments demonstrate that PAS can significantly enhance existing fast solvers in a plug-and-play manner with negligible costs. For instance, on CIFAR10, PAS requires only 12 parameters and less than 1 minute of training on a single NVIDIA A100 GPU to optimize the DDIM from 15.69 FID (NFE=10) to 4.37.
Introduction
Diffusion models are the current gold standard for generative AI, but they are notoriously slow, often requiring hundreds of denoising steps to produce high-quality images. While "fast solvers" like DDIM or DPM-Solver can reduce this to 10–20 steps, they suffer from truncation errors—mathematical drifts that accumulate when the step size is too large. Existing solutions usually involve distillation, which is computationally expensive and adds millions of parameters. This paper introduces PCA-based Adaptive Search (PAS), a "middle ground" that fixes errors of fast solvers without the heavy overhead of full model training.
Methodology
The PAS algorithm is built upon two key observations: the low-dimensional nature of sampling trajectories and the characteristic "S-curve" of cumulative truncation errors.
PCA-Based Sampling Correction
Even though image data resides in a high-dimensional space, the "path" a diffusion model takes from noise to an image typically lives in a simple 3D or 4D subspace. PAS leverages this by using Principal Component Analysis (PCA) on a small set of sample trajectories to find orthogonal unit basis vectors. Instead of training a complex neural network, PAS learns a tiny set of coordinates (coefficients) to nudge the sampling direction back toward the ground truth path.
Adaptive Search Strategy
Truncation error does not grow linearly; it follows an "S-shape," spiking in regions where the trajectory has high curvature. PAS uses a tolerance-based search to identify these specific time points. By only correcting steps where the error is most significant, the number of parameters to store is reduced from hundreds to approximately 10.
Experimental Results
The method was evaluated on standard benchmarks including CIFAR10, FFHQ, ImageNet, LSUN Bedroom, and Stable Diffusion v1.4.
Performance and Efficiency
PAS can be trained in under a minute on a single A100 GPU. On CIFAR10 at NFE=10, PAS improved DDIM’s FID from 15.69 to 4.37 using only 12 parameters. For Stable Diffusion, PAS consistently outperformed state-of-the-art solvers like DPM-Solver++ and UniPC, achieving an FID of 14.23 at NFE=10.
Geometric Consistency
A significant finding is that the geometric shape of sampling paths is remarkably consistent across a dataset. This allows coordinates learned from a small batch of samples to generalize effectively. Furthermore, PAS preserves the original ODE trajectories, maintaining smooth latent space interpolations.
Conclusion and Limitations
PAS provides a highly efficient, plug-and-play correction for diffusion sampling. However, it is dataset-specific, requiring a brief "warm-up" for new models or distributions. As a solver-dependent method, its performance is also bounded by the base solver's fundamental capabilities. Future work may explore analytical formulas for these low-dimensional curves to achieve high-quality one-step sampling.
Summary updated