Riemannian Score-Based Generative Modelling

Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, Arnaud DoucetNeurIPS 2022 · OpenReview

Introduction

Score-based Generative Models (SGMs) have emerged as a dominant force in generative AI, exhibiting remarkable empirical performance across various domains. The standard SGM framework relies on a "noising" stage—gradually adding Gaussian noise to data via a diffusion process—and a "denoising" generative stage that approximates the time-reversal of this diffusion. However, a fundamental limitation of existing SGMs is their assumption that data resides in a flat, Euclidean space ($\mathbb{R}^d$).

In reality, many scientific and real-world datasets are naturally constrained to non-flat geometries, or Riemannian manifolds. To bridge this gap, researchers have introduced Riemannian Score-based Generative Models (RSGMs), a mathematically principled extension of diffusion models designed specifically for curved spaces.

The Challenge of Curved Data

When dealing with domains like robotics (joint configurations), geoscience (global weather patterns), or protein modeling, data naturally lives on manifolds such as spheres or tori. Applying a standard "flat" diffusion model to this data is problematic. For instance, if you model Earth's climate data using standard Euclidean diffusion, the model might generate points that exist "inside" or "outside" the planet.

Tangent space and geodesic mapping on a spherical manifold.

Historically, researchers have relied on messy extrinsic projections to force these points back onto the manifold, which introduces numerical errors. Alternative intrinsic methods, like Moser Flows or Riemannian Normalizing Flows, respect the geometry but struggle to scale to high dimensions or require solving computationally expensive differential equations during training. RSGMs address these pain points by operating intrinsically from the ground up.

Methodology: Diffusion on Manifolds

To adapt the diffusion process to Riemannian manifolds, the authors overhaul both the forward and reverse stages of the SGM framework:

1. Forward Noising via Brownian Motion Instead of simply adding Gaussian noise in $\mathbb{R}^d$, RSGMs utilize Langevin dynamics or Brownian motion directly on the manifold's surface. This ensures that the progressively noisier data never leaves the valid geometric space.

2. Time-Reversal and the Stein Score The core theoretical contribution is proving a Riemannian version of the time-reversal formula. The authors demonstrate that the reverse denoising process is also a diffusion, where the "drift" is guided by the Stein score—the gradient of the log-density of the data.

3. Intrinsic Sampling with Geodesic Random Walks To generate new samples, the model must step backward through time. To avoid the accumulation of errors associated with projecting Euclidean steps back onto the manifold, RSGMs employ Geodesic Random Walks (GRWs). At each step, the model computes the required direction in the local tangent space and moves the point along the manifold's shortest path (a geodesic).

Illustration of a Geodesic Random Walk on a sphere.

4. Overcoming the Heat Kernel Hurdle Estimating the score requires calculating the transition density (the "heat kernel") on the manifold, which is notoriously difficult for compact manifolds like spheres. The authors propose two elegant workarounds:

  • Truncation: Approximating the kernel using a finite sum of the manifold's eigenfunctions.
  • Varadhan Asymptotics: A clever approximation that leverages local geodesic distances, proving highly effective for the small noise levels typical in diffusion models.

Experimental Validation

The researchers rigorously evaluated RSGMs across a variety of synthetic and real-world datasets, including Earth science data (volcanoes, earthquakes, floods, and wildfires on a sphere, $S^2$), synthetic Tori ($T^d$), the Special Orthogonal Group ($SO_3(\mathbb{R})$) for 3D rotations, and Hyperbolic Space ($\mathbb{H}^2$).

Key Results: * State-of-the-Art Performance: On Earth science datasets, RSGMs achieved negative log-likelihood scores that matched or exceeded specialized baselines. For example, on the Volcano dataset, RSGMs scored -4.92, vastly outperforming the Mixture of Kent distributions (-0.80). * High-Dimensional Scalability: Unlike Moser Flows, which suffer from exponential cost increases and collapse past 10 dimensions, RSGMs maintained high performance and constant training speeds as dimensions increased in the torus experiments. * Sampling Efficiency: RSGMs required significantly fewer neural network evaluations during sampling compared to competing methods, often achieving speedups of 10x to 70x.

Conclusion and Future Directions

Riemannian Score-based Generative Models represent a significant leap forward for modeling complex, non-Euclidean data. By keeping the entire diffusion and sampling process intrinsic to the manifold, RSGMs offer a scalable, robust, and mathematically elegant solution that outperforms previous flow-based methods.

While the current framework is highly effective for "boundaryless" manifolds (like spheres and tori), extending these techniques to manifolds with edges or highly irregular, learned geometries remains an exciting open challenge. Furthermore, the authors suggest that integrating "Schrödinger Bridges" could eventually yield even faster and higher-quality sampling paths.

Summary updated

Abstract

Score-based generative models (SGMs) are a powerful class of generative models that exhibit remarkable empirical performance.Score-based generative modelling (SGM) consists of a <code>noising'' stage, whereby a diffusion is used to gradually add Gaussian noise to data, and a generative model, which entails a</code>denoising'' process defined by approximating the time-reversal of the diffusion. Existing SGMs assume that data is supported on a Euclidean space, i.e. a manifold with flat geometry. In many domains such as robotics, geoscience or protein modelling, data is often naturally described by distributions living on Riemannian manifolds and current SGM techniques are not appropriate. We introduce here \emph{Riemannian Score-based Generative Models} (RSGMs), a class of generative models extending SGMs to Riemannian manifolds. We demonstrate our approach on a variety of compact manifolds, and in particular with earth and climate science spherical data.
BibTeX
@inproceedings{bortoli2022riemannian,
  author = {Valentin De Bortoli and Emile Mathieu and Michael Hutchinson and James Thornton and Yee Whye Teh and Arnaud Doucet},
  title = {{Riemannian Score-Based Generative Modelling}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=IAn0Y8hpUy1},
}
Figure