Abstract
Gaussian processes are an effective model class for learning unknown functions, particularly in settings where accurately representing predictive uncertainty is of key importance. Motivated by applications in the physical sciences, the widely-used Matérn class of Gaussian processes has recently been generalized to model functions whose domains are Riemannian manifolds, by re-expressing said processes as solutions of stochastic partial differential equations. In this work, we propose techniques for computing the kernels of these processes on compact Riemannian manifolds via spectral theory of the Laplace-Beltrami operator in a fully constructive manner, thereby allowing them to be trained via standard scalable techniques such as inducing point methods. We also extend the generalization from the Matérn to the widely-used squared exponential Gaussian process. By allowing Riemannian Matérn Gaussian processes to be trained using well-understood techniques, our work enables their use in mini-batch, online, and non-conjugate settings, and makes them more accessible to machine learning practitioners.
Introduction
Gaussian processes (GPs) serve as a cornerstone for probabilistic modeling, offering robust uncertainty estimates. However, their traditional application is limited to Euclidean spaces ($\mathbb{R}^d$). Many critical domains—ranging from global climate science (spherical data) to robotics (joint configurations)—involve data residing on curved Riemannian manifolds.
A naive approach to extending GPs to these domains involves replacing Euclidean distance with geodesic distance in standard kernels. However, this often violates the requirement for kernels to be positive semi-definite, a problem known as the "no-go theorem." This paper addresses this by redefining Matérn kernels through stochastic partial differential equations (SPDEs) and spectral theory.
Methodology
The core contribution is a constructive method for building valid kernels on compact manifolds.
The SPDE Connection
The authors utilize the link between Matérn GPs and SPDEs. Specifically, a Matérn GP $f$ can be viewed as the solution to a specific stochastic partial differential equation involving the Laplace-Beltrami operator $\Delta$. This shift from distance-based metrics to differential operators avoids the pitfalls of the "no-go theorem."
Spectral Theory and Constructive Kernels
On a compact manifold, the Laplace-Beltrami operator possesses a discrete spectrum of eigenpairs $(\lambda_n, f_n)$. The authors prove that the resulting Matérn kernel can be expressed as an infinite sum of these eigenfunctions, weighted by their eigenvalues:
$$k_\nu(x, x') \propto \sum_{n=0}^\infty \left(\frac{2\nu}{\kappa^2} + \lambda_n\right)^{-\nu - d/2} f_n(x)f_n(x')$$
This formulation ensures mathematical validity and allows for the use of inducing point methods, enabling scalability to large datasets through sparse approximations.
Key Insights: Topology and Global Structure
A significant advantage of the spectral approach is its ability to capture the global topology of the manifold. For instance, on a circle, the kernel must account for paths that wrap around multiple times. The spectral approach captures this global structure automatically, whereas naive geodesic kernels only remain valid locally.
Experimental Validation
The proposed framework was validated on several challenging geometries:
- Dragon Mesh: Using a complex 3D mesh, the authors demonstrated that the GP correctly identifies that two points on opposite sides of the dragon's snout are "far apart" in terms of surface geometry, even if they are physically close in 3D space.
- Pendulum Dynamics: The state space of a pendulum is a cylinder (angle $\times$ momentum). The Riemannian GP successfully learned the Hamiltonian of the system, outperforming Euclidean models by respecting the periodic nature of the angle.
- Squared Exponential Extension: The theory was successfully extended to the squared exponential kernel by treating it as a Matérn kernel with "infinite smoothness."
Conclusion
This work provides a mathematically rigorous and computationally practical bridge between Gaussian processes and Riemannian geometry. By providing explicit kernel formulas, it opens the door for Riemannian GPs to be used with modern, scalable inference techniques like mini-batch and non-conjugate training. Future directions include extending the theory to non-compact manifolds and optimizing the numerical computation of eigenfunctions for extremely high-resolution meshes.
Summary updated