Geodesic Navigation of Nonlinear Latent Manifolds

This direction is merged into another direction: manifold-constrained interpolation. Its last version stays readable and citable here.

Version 1 of 1147 papers201 conceptsfrozen 2026-10-04

In standard deep learning, we often treat the latent space of a model as a flat, Euclidean plane. When we want to transition between two points—say, morphing one face into another—we draw a straight line between them. However, the neural networks that map these latent points back to reality are highly nonlinear. This nonlinearity warps the space, turning a "flat" latent grid into a curved, complex manifold.

Recent research has shown that treating these spaces as Euclidean is a fundamental mistake that leads to "ghosting" artifacts, unrealistic intermediate states, and poor generalization. Instead, a growing body of work proposes navigating these spaces using Riemannian geometry, finding the "true" shortest paths—known as geodesics—that follow the natural curvature of the data.

The Euclidean Shortcut Problem

When you interpolate linearly between two points in a high-dimensional space, you are taking a shortcut through the "void." In a generative model, this void represents regions of low data density where the model has never seen a valid example. As Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg demonstrate in Latent Space Oddity: on the Curvature of Deep Generative Models, these straight-line paths often result in "nonsense" data because they ignore how the generator function stretches and compresses the latent space.

Imagine a map of the Earth. A straight line drawn on a flat map (Euclidean) does not represent the shortest path on the actual globe (Riemannian). Similarly, Chris Bregler and Stephen M. Omohundro found as early as 1995 that for tasks like visual speech recognition, interpolating along a learned "space of lips" ensures that intermediate frames remain physically realistic, whereas linear paths produce impossible mouth shapes (Nonlinear Manifold Learning for Visual Speech Recognition).

Defining the Metric: How the Space Curves

To find a geodesic, we first need a way to measure distance that accounts for curvature. This is done via a metric tensor. Researchers have proposed several ways to derive this "ruler" for latent spaces:

  1. Pullback Metrics: By looking at how small changes in the latent space affect the output, we can "pull back" the geometry of the data space into the latent space. Luis Augenstein and Leonel Rozo use this to create "data-aware" geodesics that follow high-density regions in hyperbolic spaces (On Probabilistic Pullback Metrics on Latent Hyperbolic Manifolds).
  2. Fisher Information Metrics: By treating data points as probability distributions, we can use the Fisher Information Metric (FIM) to measure distances. Oluwadamilola Fasina and Guy Wolf developed Neural FIM to learn these metrics directly from point clouds, allowing for smooth, manifold-aware reconstructions of biological processes (Neural FIM for learning Fisher information metrics from point cloud data).
  3. Energy-Based Metrics: Louis Béthune and Yilun Du suggest that the energy landscape of a pretrained model can act as a metric. By treating high-energy regions as "mountains" to be avoided, geodesics naturally stay in the "valleys" of real data (Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models).

Geodesic Flow Matching and Diffusion

The most recent frontier in this area involves integrating these geometric insights into generative flows and diffusion models. Standard diffusion models assume a flat noise space, but data like molecular structures or global seismic activity live on curved surfaces (like the Stiefel manifold or spheres).

Valentin De Bortoli and colleagues extended score-based models to Riemannian manifolds, allowing for the generation of data that natively respects these constraints (Riemannian Score-Based Generative Modeling). Similarly, Geodesic Flow Matching restricts the denoising process to the data's natural manifold, which Alston Lo and Austin Cheng applied to molecular assembly to ensure that intermediate states never violate physical laws like moments of inertia (Stiefel Flow Matching for Moment-Constrained Structure Elucidation).

Why It Works: Intuition over Equations

The core intuition is that the shortest mathematical path is rarely the shortest semantic path.

In a flat space, the path from "A" to "B" is a straight line. In a curved manifold, the path from "A" to "B" is a curve that stays where the data is "thickest." As Shinichi Saito and Takashi Matsubara show in their work on diffusion models, using the Jacobian of the score function to define a Riemannian metric prevents "ghosting" artifacts because the path is forced to stay tangential to the data manifold (Image Interpolation with Score-based Riemannian Metrics of Diffusion Models).

By respecting the intrinsic geometry of the data, these models don't just generate better samples—they understand the underlying structure of the world they are modeling, whether that is the hierarchical nature of robot motion (Taxonomy-aware Dynamic Motion Generation on Hyperbolic Manifolds) or the complex folding pathways of proteins (Juno Nam et al., Learning Collective Variables for Protein Folding...).

Papers in this version

Showing 60 of 147 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_geodesic_navigation_of_nonlinear_latent__a67b6ddf,
  title = {Geodesic Navigation of Nonlinear Latent Manifolds},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geodesic-navigation-of-nonlinear-latent-manifolds/rdr_a67b6ddf64494ba28a215d9154173421},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_geodesic_navigation_of_nonlinear_latent__a67b6ddf_v1,
  title = {Geodesic Navigation of Nonlinear Latent Manifolds},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geodesic-navigation-of-nonlinear-latent-manifolds/rdr_a67b6ddf64494ba28a215d9154173421/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}