Most machine learning algorithms operate under a "Flat Earth" assumption. Whether we are using Support Vector Machines or Gaussian Processes (GPs), we typically rely on kernels—functions that measure the similarity between two points—designed for flat, Euclidean space. The most famous of these, the Squared Exponential (or Gaussian) kernel, assumes that the shortest path between two points is a straight line.
However, much of the world’s data is not flat. Global weather patterns live on a sphere; robotic joint angles live on a torus; and hierarchical taxonomies are best represented in hyperbolic space. When we try to force these "curved" datasets into "flat" kernels, the mathematical foundations of our models begin to crumble.
The Geodesic Trap
A natural first instinct when working with non-Euclidean data is to simply replace the Euclidean distance in a kernel formula with the geodesic distance (the shortest path along the surface). For example, to model data on the Earth, one might use the great-circle distance instead of a straight line through the planet's core.
As Aasa Feragen, Søren Hauberg, and François Lauze demonstrate in Geodesic Exponential Kernels: When Curvature and Linearity Conflict, this "plug-and-play" approach is mathematically dangerous. For a kernel to be valid, it must be positive definite. This property ensures that the resulting covariance matrices are invertible and that the model produces physically meaningful probabilities (e.g., no negative variances).
Feragen and colleagues proved a striking negative result: a geodesic Gaussian kernel is only positive definite if the underlying space is flat. On a curved manifold, this naive substitution leads to Gram matrices with negative eigenvalues, rendering standard Bayesian optimization and regression unreliable.
Why Flat Kernels Fail on Curved Surfaces
The failure of Euclidean kernels on manifolds stems from the relationship between curvature and global topology. In Euclidean space, the Gaussian kernel's spectral transform is always positive. But on curved symmetric spaces, the interaction between the space's geometry and its group structure causes the kernel's transform to oscillate into negative values.
Nathaël Da Costa, Cyrus Mostajeran, and their colleagues show in Invariant kernels on Riemannian symmetric spaces that the Gaussian kernel is "broken" on non-compact symmetric spaces. Because these spaces contain hyperbolic planes, any kernel that fails on a hyperbolic plane—as the Gaussian does—is invalid for the entire space. This "inheritance of invalidity" means we cannot simply patch Euclidean functions; we must derive kernels from the geometry itself.
The Solution: Diffusion and Heat Kernels
If we cannot use distance-based formulas, how do we define similarity? The most robust answer lies in physics: diffusion.
Instead of asking "how far apart are these points?", we ask "how much heat would flow from point A to point B over time?". Heat follows the contours of the manifold naturally. By defining kernels as solutions to the heat equation—or more formally, using the eigenfunctions of the Laplace-Beltrami operator—researchers can construct kernels that are "born" on the manifold.
Alexander Terenin and Iskander Azangulov have championed this approach in Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I. By using the Laplace-Beltrami operator, they ensure that the resulting Gaussian Processes respect the intrinsic geometry and periodic nature of the domain (like a sphere or a rotation group) while guaranteeing positive definiteness across all length scales.
This framework has been unified in the GeometricKernels package by Peter Mostowsky and colleagues, providing a practical way to apply these "intrinsic" kernels to meshes, graphs, and manifolds.
Specialized Kernels for Specific Geometries
While the heat kernel is a universal solution, specific geometries often benefit from tailored mathematical structures:
- Hyperbolic Spaces: For hierarchical data, Pengfei Fang and Mehrtash Harandi have developed kernels that map hyperbolic representations into Reproducing Kernel Hilbert Spaces (RKHS) without losing the tree-like structure of the data (Kernel Methods in Hyperbolic Spaces).
- SPD Manifolds: In computer vision, data often takes the form of Symmetric Positive Definite (SPD) matrices (like covariance descriptors). Richard Hartley and Sadeep Jayasumana demonstrated that using the Log-Euclidean metric allows for valid Gaussian kernels on these manifolds, enabling more accurate medical imaging and texture analysis (Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices).
- Constrained Domains: When data is restricted to "U-shaped" regions or manifolds with holes, Lizhen Lin and David B. Dunson proposed intrinsic Gaussian processes (in-GPs). These use Brownian motion simulations to prevent information from "leaking" across physical barriers that are close in Euclidean space but far apart along the manifold (Intrinsic Gaussian processes on complex constrained domains).
Why It Works: Respecting Intrinsic Topology
The core intuition behind this research direction is geometric inductive bias. A standard GP might predict that two points on opposite sides of a thin wall are highly correlated because they are physically close. A geometry-aware GP, however, understands that the "distance" between them must be measured along the hallway and through the door.
By moving from extrinsic Euclidean distances to intrinsic geometric operators, we allow machine learning models to "feel" the shape of the data. As Emilio Porcu and Chris J. Oates highlight in their "grand tour" of the Matérn model, this transition is not just a mathematical nicety—it is the bridge required to make Gaussian Processes scalable and valid for the massive, non-Euclidean datasets of the modern era.