Maximum Likelihood Training for Score-based Diffusion ODEs by High Order Denoising Score Matching

Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

Introduction

Score-based generative models (SGMs) have become a cornerstone of modern AI, powering state-of-the-art tools like DALL-E and Stable Diffusion. These models operate by learning the "score function"—the gradient of the log-density of the data. Once learned, this score can be used to generate samples via a Stochastic Differential Equation (SDE) or a deterministic Ordinary Differential Equation (ODE).

The ODE formulation (ScoreODE) is particularly attractive because it allows for exact likelihood evaluation, meaning we can calculate exactly how well the model fits the data distribution in terms of bits-per-dimension (bpd). However, a critical theoretical gap exists: training a model to match the first-order score (the standard approach) does not necessarily maximize the likelihood of the resulting ODE. Consequently, models trained this way often fail to estimate the density of even simple distributions correctly.

This paper bridges this gap by proving that standard first-order score matching is insufficient for maximizing the likelihood of the associated Probability Flow ODE. To solve this, the authors introduce a novel High-Order Denoising Score Matching (DSM) algorithm that incorporates second and third-order derivatives, significantly improving density estimation without sacrificing world-class image generation quality.

Methods

The authors' approach is rooted in a rigorous mathematical analysis of the KL divergence (a measure of how two distributions differ) between the true data distribution and the ScoreODE.

  1. The Theoretical Gap: The authors mathematically prove that the negative log-likelihood of the ODE is bounded by the errors in the first, second, and third-order score matching. Essentially, to accurately capture the density, a model must match not just the gradient, but also the curvature (Hessian) and the rate of change of that curvature.
  2. High-Order DSM: To close this gap, they propose a new training objective that explicitly includes terms for these higher-order derivatives.
Relationship between SDE/ODE paths and score matching objectives.
  1. Efficiency via Trace Estimators: Computing full Hessian matrices for high-dimensional data like images is computationally prohibitive ($O(d^2)$ complexity). To overcome this, the authors utilize Skilling-Hutchinson trace estimators alongside efficient "Jacobian-vector-product" computations in JAX. This reduces the computational cost to $O(d)$, making high-order training feasible for real-world datasets.
  2. Stop-Gradients: To ensure training stability and compatibility with existing frameworks, "stop-gradient" operations are applied to lower-order terms when optimizing the higher-order components.

Key Insights and Ideas

  • First-Order is Not Enough: The most striking insight is that a model can match the first-order score perfectly but still produce a density map that fails to represent the original data. The ODE's likelihood is highly sensitive to the divergence of the score function.
  • The "Aha" Moment on Curvature: By explicitly training the model to match the second and third-order properties of the data distribution, the ODE is "forced" to follow the true density more closely.

This phenomenon is clearly visualized using a 1-D Mixture of Gaussians. As seen below, the standard first-order model produces severe artifacts in its density estimation, whereas the proposed third-order model perfectly recovers the smooth, multi-modal structure of the true data.

Comparison of density estimation on a 1D Mixture of Gaussians.
  • Compatibility: Importantly, the optimal solution for the high-order objective remains the true data score function. This means the exact same model can be used for both high-quality SDE sampling and high-accuracy ODE likelihood evaluation.

Datasets and Key Results

The researchers evaluated their method on a mix of synthetic benchmarks (1-D Mixture of Gaussians, 2-D Checkerboard) and high-dimensional image datasets (CIFAR-10, ImageNet 32x32).

The quantitative improvements in likelihood (measured in bits-per-dimension, where lower is better) are significant:

  • CIFAR-10: On the "Variance Exploding" (VE) deep model, likelihood improved from 3.45 bpd (standard first-order) to 3.27 bpd (third-order).
  • ImageNet 32x32: Likelihood improved from 4.21 bpd to 4.03 bpd.
  • Sample Quality: Crucially, the models maintained high sample quality. On CIFAR-10, the FID score (a measure of image realism) remained excellent, proving that optimizing for likelihood does not have to come at the cost of visual crispness.
  • Visual Density: On synthetic data like the 2-D Checkerboard, the third-order model perfectly recovered the multi-modal structure that first-order models blurred or missed entirely.

Limitations and Future Work

While highly effective, the proposed method does introduce computational overhead. High-order training is more expensive; the paper notes that third-order training takes about 1.5x more time per iteration and requires roughly double the memory compared to standard first-order training. Additionally, there are minor challenges with numerical stability when the diffusion time $t$ is very close to zero, requiring a small offset ($\epsilon$) during training.

Future work could explore extending these high-order objectives to latent diffusion models or other specialized diffusion variants, potentially unlocking better likelihood estimation across an even broader range of generative architectures.

Summary updated

Abstract

Score-based generative models have excellent performance in terms of generation quality and likelihood. They model the data distribution by matching a parameterized score network with first-order d...
BibTeX
@inproceedings{lu2022maximum,
  author = {Cheng Lu and Kaiwen Zheng and Fan Bao and Jianfei Chen and Chongxuan Li and Jun Zhu},
  title = {{Maximum Likelihood Training for Score-based Diffusion ODEs by High Order Denoising Score Matching}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=BaslWPTkWRH},
}
Figure