SciRE-Solver: Accelerating Diffusion Models Sampling by Score-integrand Solver with Recursive Difference

Shigui Li, Wei Chen, Delu Zeng

Abstract

Diffusion models (DMs) have made significant progress in the fields of image, audio, and video generation. One downside of DMs is their slow iterative process. Recent algorithms for fast sampling are designed from the perspective of differential equations. However, in higher-order algorithms based on Taylor expansion, estimating the derivative of the score function becomes intractable due to the complexity of large-scale, well-trained neural networks. Driven by this motivation, in this work, we introduce the recursive difference (RD) method to calculate the derivative of the score function in the realm of DMs. Based on the RD method and the truncated Taylor expansion of score-integrand, we propose SciRE-Solver with the convergence order guarantee for accelerating sampling of DMs. To further investigate the effectiveness of the RD method, we also propose a variant named SciREI-Solver based on the RD method and exponential integrator. Our proposed sampling algorithms with RD method attain state-of-the-art (SOTA) FIDs in comparison to existing training-free sampling algorithms, across both discrete-time and continuous-time pre-trained DMs, under various number of score function evaluations (NFE). Remarkably, SciRE-Solver using a small NFEs demonstrates promising potential to surpass the FID achieved by some pre-trained models in their original papers using no fewer than 1,000 NFEs. For example, we reach SOTA value of 2.40 FID with 100 NFE for continuous-time DM and of 3.15 FID with 84 NFE for discrete-time DM on CIFAR-10, as well as of 2.17 (2.02) FID with 18 (50) NFE for discrete-time DM on CelebA 64x64.

Introduction

Diffusion Models (DMs) have revolutionized generative AI, but they suffer from a significant "speed tax," typically requiring hundreds or thousands of iterative denoising steps. Mathematically, this denoising process is equivalent to solving an Ordinary Differential Equation (ODE). While advanced numerical ODE solvers can accelerate this, high-order solvers (e.g., those based on Taylor expansions) require derivatives of the "score function"—the neural network itself. Calculating exact derivatives for massive, complex networks is computationally prohibitive, and traditional Finite Difference (FD) approximations are often numerically unstable in this context. SciRE-Solver addresses this challenge by providing accurate, stable higher-order information without breaking the computational budget.

Methods

The core innovation of this work is the Recursive Difference (RD) Method. Instead of standard finite differences, the RD method recursively extracts "hidden" lower-order derivative information from the higher-order terms of the Taylor expansion of the score-integrand. This provides a stable, mathematically grounded estimation of how the model's output changes over time.

The Recursive Difference (RD) method extracts derivative information from the Taylor expansion of the score-integrand.

Building on this, the authors propose two primary solvers:

  1. SciRE-Solver: Integrates the RD method within a score-integrand framework, providing a convergence order guarantee.
  2. SciREI-Solver: A variant combining the RD method with an exponential integrator, a technique popular in solvers like DPM-Solver.

Additionally, the authors propose Parametrizable Time Trajectories (NSR-type and Sigmoid-type). Rather than stepping through time linearly, these schedules allow the solver to spend more effort at critical parts of the denoising process, further boosting efficiency.

Results

The solvers were evaluated across a wide range of standard benchmarks, including CIFAR-10, CelebA 64x64, and high-resolution ImageNet and LSUN datasets. SciRE-Solver consistently outperformed established baselines like DDIM and DPM-Solver, particularly in low-NFE regimes.

Visual comparison showing SciRE-Solver's superior image quality at low NFE (10-20) compared to DDIM and DPM-Solver.

Key quantitative highlights include:

  • CIFAR-10: Achieved a SOTA FID of 2.40 with 100 NFE for continuous-time models.
  • CelebA 64x64: Reached a SOTA FID of 2.17 in just 18 steps, and 2.02 with 50 steps.
  • Efficiency: In many cases, SciRE-Solver achieved better image quality at 12–20 steps than competing solvers achieved at higher step counts. It often surpasses the quality of original models that used 1,000+ steps.

Qualitatively, visual comparisons show that SciRE-Solver produces sharper details and fewer "noisy" artifacts in low-step regimes compared to previous training-free methods. It also demonstrates significant robustness to the choice of time schedule, maintaining trajectory accuracy where other solvers might fail.

Conclusion

SciRE-Solver addresses the bottleneck of derivative estimation in high-order diffusion sampling. By recycling information through the RD method, it achieves state-of-the-art performance without the need for additional training. While the current work focuses on 2nd and 3rd-order solvers, future research may explore higher-order extensions and further optimization of time trajectories to handle highly non-linear denoising paths.

Summary updated

Abstract

Diffusion models (DMs) have made significant progress in the fields of image, audio, and video generation. One downside of DMs is their slow iterative process. Recent algorithms for fast sampling are designed from the perspective of differential equations. However, in higher-order algorithms based on Taylor expansion, estimating the derivative of the score function becomes intractable due to the complexity of large-scale, well-trained neural networks. Driven by this motivation, in this work, we introduce the recursive difference (RD) method to calculate the derivative of the score function in the realm of DMs. Based on the RD method and the truncated Taylor expansion of score-integrand, we propose SciRE-Solver with the convergence order guarantee for accelerating sampling of DMs. To further investigate the effectiveness of the RD method, we also propose a variant named SciREI-Solver based on the RD method and exponential integrator. Our proposed sampling algorithms with RD method attain state-of-the-art (SOTA) FIDs in comparison to existing training-free sampling algorithms, across both discrete-time and continuous-time pre-trained DMs, under various number of score function evaluations (NFE). Remarkably, SciRE-Solver using a small NFEs demonstrates promising potential to surpass the FID achieved by some pre-trained models in their original papers using no fewer than $1000$ NFEs. For example, we reach SOTA value of $2.40$ FID with $100$ NFE for continuous-time DM and of $3.15$ FID with $84$ NFE for discrete-time DM on CIFAR-10, as well as of $2.17$ (2.02) FID with $18$ (50) NFE for discrete-time DM on CelebA 64$\times$64.
BibTeX
@misc{li2023scire,
  author = {Shigui Li and Wei Chen and Delu Zeng},
  title = {{SciRE-Solver: Accelerating Diffusion Models Sampling by Score-integrand Solver with Recursive Difference}},
  year = {2023},
  eprint = {2308.07896},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.07896},
}
Figure