Many machine learning algorithms share a foundational assumption: the underlying data space is flat and Euclidean. In this idealized world, the shortest distance between two points is always a straight line. However, real-world data rarely conforms to this assumption. Whether dealing with the orientations of a robot arm, the complex geometry of a human heart, or high-dimensional perceptual images, data often lives on curved, non-Euclidean surfaces known as manifolds.
When we force Euclidean metrics onto curved spaces, our models suffer from geometric distortion. To fix this, a major research direction has emerged focused on constructing intrinsic, manifold-aware kernels. These kernels allow probabilistic models, optimization frameworks, and classifiers to respect the true underlying geometry of the data.
The Failure of Flat Assumptions
To understand why standard methods fail, consider a dataset shaped like a horseshoe or a tightly rolled sheet of paper (a Swiss roll). If we measure the distance between two points on opposite sides of the fold using standard Euclidean distance, they appear close together.
In a foundational paper, Joshua B. Tenenbaum introduced Isomap in Mapping a Manifold of Perceptual Observations, demonstrating that for high-dimensional data, we must replace straight-line ambient distances with geodesic distances—the shortest path traveling strictly along the surface of the manifold. As shown in Extended Isomap for Pattern Classification by Ming-Hsuan Yang et al., respecting this intrinsic geometry prevents classification errors that occur when distinct classes appear close in Euclidean space purely due to manifold curvature.
The Naive Solution and Its Breakdown
If geodesic distances capture the true shape of the data, a natural next step is to inject them directly into standard kernel methods. For instance, we might take the classic Gaussian Radial Basis Function (RBF) kernel:
$$k(x, x') = \exp(-\gamma \|x - x'\|^2)$$
and simply substitute the Euclidean distance $\|x - x'\|$ with the geodesic distance $d_g(x, x')$.
This strategy of "geodesic substitution" has been applied across various domains. For example, Bruno Pelletier adapted kernel classification rules to curved spaces in A kernel-based classifier on a Riemannian manifold. Similarly, in A Framework for Shape Analysis via Hilbert Space Embedding, Sadeep Jayasumana et al. utilized pairwise geodesic relationships to avoid the distortions introduced by projecting shapes onto flat tangent spaces.
However, this naive substitution breaks a core mathematical requirement of kernel methods: positive definiteness. For a kernel to be valid—meaning it defines a legitimate Reproducing Kernel Hilbert Space (RKHS) or a valid covariance matrix for a Gaussian Process—the kernel matrix for any set of points must never yield negative eigenvalues.
Aasa Feragen et al. exposed this fundamental limitation in Geodesic exponential kernels: When curvature and linearity conflict. They proved that if a manifold possesses any curvature, the geodesic exponential kernel is guaranteed to be non-positive definite at certain length scales. While the authors later noted in Open Problem: Kernel methods on manifolds and metric spaces. What is the probability of a positive definite geodesic exponential kernel? that these kernels can occasionally work for specific finite datasets, relying on them introduces severe mathematical instability, particularly when quantifying uncertainty.
The Rigorous Path: The Laplace-Beltrami Operator
To build mathematically sound, geometry-aware models, researchers had to stop patching Euclidean formulas and instead define kernels intrinsically using differential geometry.
The solution lies in the spectral theory of the Laplace-Beltrami operator ($\Delta_M$), which generalizes the standard Laplacian (used to compute gradients and divergence) to curved manifolds. In flat Euclidean space, standard kernels like the Matérn or squared exponential kernels can be defined via Fourier transforms, which fundamentally rely on the eigenfunctions of the standard Laplacian (sine and cosine waves). On a curved manifold, we can mirror this process by finding the eigenvalues ($\lambda_i$) and eigenfunctions ($\phi_i$) of the Laplace-Beltrami operator:
$$\Delta_M \phi_i = \lambda_i \phi_i$$
In Matérn Gaussian Processes on Riemannian Manifolds, Viacheslav Borovitskiy et al. leveraged this spectral decomposition to construct mathematically valid Matérn kernels directly on compact Riemannian manifolds. By defining the kernel as a convergent sum over these eigenfunctions, the resulting covariance matrices are guaranteed to be positive-definite across all length scales.
An intuitive way to visualize this is through heat diffusion. A heat kernel dictates how thermal energy spreads across a surface over time. Because heat cannot jump across the empty ambient space separating two folds of a manifold—it must flow continuously along the surface—a heat-kernel-based covariance naturally respects the manifold's topology.
This formulation completely prevents uncertainty leakage. In a standard Gaussian Process, a model might mistakenly assume that a data point on one side of a horseshoe fold tells us something certain about a point on the opposite side because they are spatially close. An intrinsic, Laplace-Beltrami-derived kernel forces the correlation to travel along the surface distance, ensuring that uncertainty quantification remains accurate and bounded by the true geometry.
Why Geometry Awareness Matters in Practice
Building kernels from the intrinsic geometry of the space unlocks significant performance gains in complex physical and data-driven domains.
Data-Efficient Bayesian Optimization
In Bayesian Optimization (BO), a Gaussian Process acts as a surrogate model to guide the search for an optimal solution. If the search space consists of non-Euclidean constraints—such as robot joint angles, orientations, or symmetric positive-definite matrices—a standard Euclidean surrogate will waste evaluations exploring physically or anatomically impossible regions.
Noémie Jaquier et al. demonstrated in Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels and Bayesian Optimization Meets Riemannian Manifolds in Robot Learning that geometry-aware BO dramatically improves data efficiency when teaching robots complex behaviors. Similarly, Paris Perdikaris et al. applied this framework in Learning cardiac activation maps from 12-lead ECG with multi-fidelity Bayesian optimization on manifolds to locate abnormal heartbeats, forcing the optimization algorithm to respect the actual physical geometry of the heart.
Modeling Complex Physical Structures
Whether tracking uncertainty across biological surfaces (Gaussian Process Landmarking on Manifolds by Tingran Gao et al.) or modeling random fields on thin-shell engineering structures like turbine blades (Stochastic PDE representation of random fields for large-scale Gaussian process regression and statistical finite element analysis by Kim Jie Koh et al.), integrating the Laplace-Beltrami operator ensures that spatial correlations match physical reality.
Global Consistency on Spheres and Hyperspheres
For global spatial data or directional statistics (such as wind patterns on Earth), approximating distances using flat, extrinsic kernels introduces severe distortion over long distances. As emphasized by Simon Hubbert et al. in Sobolev Spaces, Kernels and Discrepancies over Hyperspheres, modeling data on curved manifolds requires intrinsic kernels that natively align with the topology of the space to maintain consistency across the entire domain.
By shifting from extrinsic, flat approximations to intrinsic, operator-valued definitions, modern kernel methods can accurately navigate the curved geometries inherent to the physical world.
Key Foundational Work
Extended Isomap for Pattern Classification (2002) — contributed 1 concept elements to this direction.
Mapping a Manifold of Perceptual Observations (1997) — contributed 1 concept elements to this direction.