Generative Fractional Diffusion Models

Gabriel Nobis, Marco Aversa, Maximilian Springenberg, Michael Detzel, Stefano Ermon, Shinichi Nakajima, Roderick Murray-Smith, Sebastian Lapuschkin, Christoph Knochenhauer, Luis Oala, Wojciech Samek

Introduction

Standard score-based generative models, such as Denoising Diffusion Probabilistic Models (DDPM) and Stable Diffusion, typically rely on Brownian Motion (BM) to transform data into noise. While highly effective, BM is characterized by independent increments and "light-tailed" distributions. These inherent properties can limit sample diversity and lead to slow convergence or mode collapse, particularly on imbalanced or complex datasets.

This paper introduces Generative Fractional Diffusion Models (GFDM), the first continuous-time score-based models to replace standard Brownian motion with fractional Brownian motion (fBM). By introducing the Hurst index ($H$), researchers gain a tunable parameter to control the "roughness" and "memory" of the diffusion process, interpolating between highly irregular paths ($H < 0.5$) and smooth, long-memory paths ($H > 0.5$).

Methods: Making fBM Tractable

The primary challenge with fBM is its non-Markovian nature; unlike standard BM, the future state of an fBM process depends on its entire history. This makes the "reverse-time" math required for generative modeling extremely complex. To overcome this, the authors propose a Markovian Approximation (MA-fBM).

Visualizing the forward and reverse dynamics of GFDM. The Hurst index $H$ allows for interpolation between rough ($H < 0.5$) and smooth ($H > 0.5$) diffusion paths.

As illustrated in Figure 1, the fBM process is approximated using a linear combination of $K$ Ornstein-Uhlenbeck (OU) processes. This transformation maps a non-Markovian problem into an augmented Markovian state space. Crucially, the authors derive a novel Augmented Score Matching loss function. This derivation proves that the neural network only needs to model the original data dimension, as the scores of the augmenting OU processes are known and act as guidance for the unknown data score. This ensures that GFDM can be trained without increasing the parameter count of the underlying model.

Experimental Results

The GFDM framework was evaluated on the MNIST and CIFAR-10 benchmarks, comparing performance against standard Variance Exploding (VE) and Variance Preserving (VP) diffusion baselines.

Image Quality and Sampling Efficiency

The experiments revealed that the super-diffusive regime ($H = 0.9$) consistently yields superior results. On MNIST, GFDM achieved a state-of-the-art FID of 0.72, significantly outperforming the Brownian VP baseline of 1.44. On CIFAR-10, the model reached an FID of 3.77, compared to 4.85 for the baseline.

Averaged FID vs. Number of Function Evaluations (NFE). Fractional models (FVP) achieve lower FID scores and better convergence compared to purely Brownian baselines.

As shown in Figure 2, GFDM (labeled as FVP) not only achieves higher quality but also demonstrates superior sampling efficiency. The fractional models often reach peak quality with fewer function evaluations (NFEs) than standard models, suggesting they are more computationally efficient during inference.

Diversity and Distribution Coverage

Beyond image quality, GFDM significantly improves sample diversity. Using the Vendi Score ($VS_p$) as a metric, the authors found that increasing the number of augmenting processes ($K$) directly correlates with increased pixel-wise diversity. Furthermore, improved Recall scores across almost all image classes indicate that fractional dynamics help the model better cover the entire data distribution and avoid mode collapse.

Conclusion

Generative Fractional Diffusion Models provide a powerful generalization of standard diffusion by leveraging the unique properties of fractional Brownian Motion. By tuning the Hurst index, GFDM achieves better image quality, higher diversity, and improved distributional coverage with minimal computational overhead (increasing training time by only ~0.011 seconds per step). Future work will explore the scalability of these models to high-resolution latent diffusion tasks and automated methods for selecting the optimal Hurst index for different data modalities.

Summary updated

Abstract

We introduce the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics. Although diffusion models have excelled at capturing data distributions, they still suffer from various limitations such as slow convergence, mode-collapse on imbalanced data, and lack of diversity. These issues are partially linked to the use of light-tailed Brownian motion (BM) with independent increments. In this paper, we replace BM with an approximation of its non-Markovian counterpart, fractional Brownian motion (fBM), characterized by correlated increments and Hurst index $H \in (0,1)$, where $H=1/2$ recovers the classical BM. To ensure tractable inference and learning, we employ a recently popularized Markov approximation of fBM (MA-fBM) and derive its reverse time model, resulting in generative fractional diffusion models (GFDMs). We characterize the forward dynamics using a continuous reparameterization trick and propose an augmented score matching loss to efficiently learn the score-function, which is partly known in closed form, at minimal added cost. The ability to drive our diffusion model via fBM provides flexibility and control. $H \leq 1/2$ enters the regime of rough paths whereas $H>1/2$ regularizes diffusion paths and invokes long-term memory as well as a heavy-tailed behaviour (super-diffusion). The Markov approximation allows added control by varying the number of Markov processes linearly combined to approximate fBM. Our evaluations on real image datasets demonstrate that GFDM achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID, offering a promising alternative to traditional diffusion models.
BibTeX
@inproceedings{nobis2024generative,
  author = {Gabriel Nobis and Marco Aversa and Maximilian Springenberg and Michael Detzel and Stefano Ermon and Shinichi Nakajima and Roderick Murray-Smith and Sebastian Lapuschkin and Christoph Knochenhauer and Luis Oala and Wojciech Samek},
  title = {{Generative Fractional Diffusion Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=B9qg3wo75g},
}
Figure