Standard generative models—such as Normalizing Flows, Diffusion Models, and Flow Matching—typically operate under a flat assumption: they treat data as if it resides in a Euclidean space ($\mathbb{R}^d$). However, many real-world datasets are intrinsically constrained by geometry. Geospatial coordinates exist on a sphere; molecular torsion angles live on a torus; robotic joint configurations and brain imaging data often reside on complex non-Euclidean manifolds or spaces of Symmetric Positive Definite (SPD) matrices.
When standard Euclidean models are applied to these constrained spaces, they suffer from a fundamental geometric mismatch. A Euclidean model does not respect the boundaries or curvature of a manifold, leading it to assign probability mass to invalid regions (such as points outside a sphere or mathematically impossible rotation states). As noted by Elamvazhuthi et al., ignoring these underlying geometric structures results in the generation of physically or mathematically impossible states.
To solve this, a rapidly growing body of research focuses on building generative models that operate natively within Riemannian geometries, ensuring that every step of the generation process strictly respects the manifold's constraints.
Continuous Flows on Curved Surfaces
Normalizing flows generate complex probability distributions by transforming a simple base distribution through an invertible mapping. As reviewed by Kobyzev et al., this framework provides exact density estimation and efficient sampling, but adapting it to non-Euclidean geometries requires redefining how data moves.
Instead of moving arbitrarily across a flat space, continuous normalizing flows (CNFs) on manifolds define trajectories via Ordinary Differential Equations (ODEs) governed by a vector field. The core insight of Riemannian Continuous Normalizing Flows (RCNFs), introduced by Mathieu et al., is that the vector field must natively lie within the tangent space of the manifold at any given point. By integrating these tangent vectors, the resulting trajectory is mathematically guaranteed to remain on the manifold surface without requiring ad-hoc projections that distort probability densities.
Standard CNF (Euclidean): dx/dt = f(x, t) where f(x, t) ∈ ℝᵈ
Riemannian CNF: dx/dt = f(x, t) where f(x, t) ∈ TₓM (Tangent Space)
To make these models practical, researchers must overcome significant computational hurdles. For instance, calculating the change in probability density requires computing the Jacobian determinant of the transformation, which typically scales at a prohibitive $O(d^3)$ complexity. Negri et al. demonstrated that for "star-like" manifolds—shapes like spheres or the probability simplex where every boundary point is visible from a central origin—exploiting the intrinsic geometry can reduce this cost to $O(d^2)$, enabling high-fidelity variational inference.
Furthermore, training CNFs traditionally requires solving these differential equations at every training step. To scale this process, Ben-Hamu et al. introduced a "Probability Path Divergence" (PPD) objective, which allows matching normalizing flows to target probability paths on manifolds without explicitly simulating the ODEs during training.
The Challenge of Manifold Diffusion and the Heat Kernel
Denoising diffusion models generate data by reversing a progressive noising process. In Euclidean space, this noising process is straightforward: you repeatedly add Gaussian noise. On a Riemannian manifold, however, the equivalent of Gaussian diffusion is Brownian motion, which is governed by the Laplace-Beltrami operator. The probability density of this process over time is known as the heat kernel.
As Huang et al. demonstrated with Riemannian Diffusion Models (RDMs), diffusing data natively via the heat kernel prevents manifold violations. However, the heat kernel rarely has a closed-form solution on curved surfaces, often requiring expensive infinite-series approximations that prevent models from scaling to high dimensions.
To unlock high-dimensional applications, researchers have developed several workarounds to bypass or efficiently estimate the heat kernel:
- Symmetric Space Estimators: Lou et al. leveraged the algebraic properties of Riemannian symmetric spaces to create fast, highly accurate heat kernel estimators, allowing diffusion models to scale to hundreds of dimensions in fields like lattice Quantum Chromodynamics.
- Euclidean Projection: Instead of computing the intrinsic heat kernel, Mangoubi et al. utilized a projection of Euclidean Brownian motion onto symmetric manifolds. This approach achieves $O(1)$ gradient evaluations per training step, providing polynomial-time sampling guarantees.
- Bridge Processes: Jo et al. formulated the diffusion process as a mixture of Riemannian bridge processes. By utilizing a "two-way bridge matching" objective, they eliminated the need for both heat kernel approximations and divergence calculations, accelerating training significantly.
Accelerating Generation: Flow Matching and Consistency Models
While diffusion models provide high sample quality, their iterative nature makes them slow. To address this, researchers have extended alternative paradigms like Flow Matching and Consistency Models to curved geometries.
Chen et al. introduced Riemannian Flow Matching (RFM), a framework that defines vector fields to generate straight paths on general geometries using a "premetric" approach, avoiding expensive simulations. When the underlying manifold is not known explicitly but must be inferred from data, Bekkers et al. developed Pullback Flow Matching, which pulls back the geometry of a learned data manifold into a latent space to perform exact geometric modeling.
To achieve single-step generation, Cheng et al. introduced the Riemannian Consistency Model (RCM). By enforcing consistency along the intrinsic geometric trajectories of the manifold, RCMs can map noise to data in a single step without violating mathematical or physical boundaries.
The theoretical foundations of these models are also catching up to their empirical success. Xu et al. provided the first proof of polynomial convergence for Riemannian diffusion models, demonstrating that these models can achieve high accuracy with a polynomially small step size even when relaxing strict data-smoothness assumptions.
Specialized Geometries and Infinite Dimensions
As the field matures, researchers are moving beyond simple spheres and tori to accommodate highly specialized mathematical structures.
For applications in brain imaging and traffic flow modeling, data frequently takes the form of Symmetric Positive Definite (SPD) matrices. Li et al. developed SPD-DDPM, which redefines standard diffusion operations to respect the unique affine-invariant geometry of SPD spaces. When the task requires translating data between entirely different geometric domains, Chakraborty et al. proposed ManifoldGLOW, a flow-based model designed specifically for learning manifold-to-manifold mappings.
[Data on Manifold A] ──(ManifoldGLOW)──> [Data on Manifold B]
Finally, this geometric paradigm is expanding beyond finite-dimensional spaces. Mathieu introduced Geometric Neural Diffusion Processes to model continuous stochastic fields (such as global wind patterns) as distributions over functions that natively respect spherical and rotational symmetries. To unify these diverse directions, Shi et al. proposed Denoising Markov Models (DMMs), a generalized framework that extends score matching and diffusion principles to arbitrary non-Euclidean state spaces, including discrete, Riemannian, and simplex domains.
Key Foundational Work
Riemannian Continuous Normalizing Flows (2020) — contributed 1 concept elements to this direction.
Manifold-Valued Image Generation with Wasserstein Generative Adversarial Nets. (2019) — contributed 1 concept elements to this direction.