In high-dimensional machine learning, we often assume that data points are vectors in a flat, Euclidean space. We measure the similarity between two points by drawing a straight line between them—the "as the crow flies" distance. However, most real-world data—whether it is the pixel space of natural images, the activation patterns of a brain, or the embeddings of a language model—does not fill this high-dimensional volume uniformly. Instead, it concentrates on low-dimensional, curved surfaces known as manifolds.
When data lives on a manifold, Euclidean distance becomes a deceptive metric. It suggests that two points are close because the straight line between them is short, even if that line passes through a "void" where no valid data can exist. To truly understand the relationship between points, we must instead measure distance geodesically—following the shortest path along the data's intrinsic surface.
The Failure of Flat Geometry
The fundamental problem with Euclidean metrics in high dimensions is the "short-circuit" effect. As noted by Ricardo Dominguez-Olmedo and Bernhard Schölkopf in On Data Manifolds Entailed by Structural Causal Models, standard distances fail to account for the curvature of the data manifold. This leads to models that suggest "impossible" transitions—for example, a counterfactual explanation that moves a data point through a region of zero probability to reach a target state.
This mismatch is particularly visible in specialized data structures like Symmetric Positive Definite (SPD) matrices, used to model brain connectivity. Dehan Kong and Qiang Sun argue in Modeling Symmetric Positive Definite Matrices with An Application to Functional Brain Connectivity that using flat maps for these curved spaces is as distortive as using a flat paper map to measure the distance between two cities on a globe.
Building the Geodesic Intuition
If the Euclidean distance is a crow flying over a mountain range, the geodesic distance is an ant walking over the peaks and valleys. To calculate this, researchers have developed several ways to "sense" the surface of the data:
- Local Approximations: Didong Li and David B. Dunson introduced Spherelets, which use local spherical patches rather than flat planes to approximate the manifold. This captures curvature more accurately than linear methods like PCA.
- Diffusion and Random Walks: By simulating a particle diffusing across a graph of data points, we can infer the manifold's shape. Alexander Tong and Guy Wolf utilize this in Diffusion Earth Mover's Distance and Distribution Embeddings to compare entire datasets by following their intrinsic geometric structures.
- Graph-Based Paths: In Clustering with the Connectivity Kernel, Bernd Fischer and Volker Roth show that by redefining distance based on the "weakest link" in a path between points, we can transform complex, winding clusters into simple shapes that standard algorithms like K-means can finally understand.
Geometry-Aware Optimization
Once we accept that the world is curved, we must change how we move through it. This is the core of Riemannian Metric Learning (RML). As reviewed by Josua Sassen in Riemannian Metric Learning: Closer to You than You Imagine, RML replaces the uniform "grid" of Euclidean space with a metric tensor that varies depending on where you are on the manifold.
This has profound implications for how AI learns:
- Robotics: In Bayesian Optimization Meets Riemannian Manifolds in Robot Learning, Noémie Jaquier and Leonel Dario Rozo demonstrate that robots learn complex orientations much faster when the optimizer knows it is working on a manifold (like a sphere) rather than a flat 3D space.
- Optimal Transport: Standard Optimal Transport (OT) often "cuts corners." Geodesic Sinkhorn by Alexander Tong et al. forces the transport of "mass" to stay on the manifold, leading to more realistic interpolations between data points.
- Generative Models: Byoung-Tak Zhang and Jiseob Kim highlight in Data Interpolations in Deep Generative Models under Non-Simply-Connected Manifold Topology that if a manifold has "holes" (non-simply-connected), a straight-line interpolation will fail. Geometry-aware paths must "go around" these holes to maintain data realism.
The Semantic Bridge
Perhaps the most intuitive argument for geodesic metrics comes from retrieval. In Using Text to Teach Image Retrieval, Haoyu Dong and Guillermo Sapiro explain that measuring distance as a straight line through a high-dimensional void fails to reflect the semantic path between items. By using text to "fill the gaps" in the image manifold, they allow for geodesic measurements that follow the logical, incremental transitions between concepts.
By shifting from the "crow's" Euclidean view to the "ant's" geodesic view, machine learning models stop fighting the natural shape of data and start leveraging it. As Georgios Batzolis and colleagues suggest in Score-based pullback Riemannian geometry, the goal is no longer just to "unroll" the manifold into a flat plane, but to learn to navigate its curves directly.