When we train a deep generative model, such as a Variational Autoencoder (VAE) or a Generative Adversarial Network (GAN), we map a complex, high-dimensional data distribution into a lower-dimensional latent space. By default, standard deep learning frameworks treat this latent space as a flat, Euclidean plane. Under this assumption, the shortest and most natural path between two latent points is a simple straight line.
However, this Euclidean assumption is an illusion. The generator function that maps latent codes back to the observation space is highly non-linear. It stretches some regions of the latent space and compresses others, warping the underlying geometry. When we perform standard linear interpolation, the straight-line trajectory frequently cuts through "dead zones"—low-density regions where the model has never seen data. This results in blurry, unrealistic, or semantically jarring intermediate samples.
To fix this, a growing body of research replaces flat Euclidean trajectories with geometry-aware paths that respect the true, curved data manifold.
The Pullback Metric: Revealing the True Geometry
To navigate a warped latent space accurately, we must treat it as a Riemannian manifold—a curved mathematical space where distances are measured locally. The foundational tool for uncovering this geometry is the pullback metric.
Imagine walking across a rubber sheet that has been unevenly stretched. A step of fixed length on the sheet might cover a huge distance in some areas but barely move you in others. The pullback metric mathematically quantifies this distortion. By taking the Jacobian of the generator function—which calculates how a tiny change in latent coordinates affects the generated output—we can project the metric of the high-dimensional observation space back into the low-dimensional latent space.
As Georgios Arvanitidis et al. demonstrate, this local metric tensor accounts for the non-linearity of the generator, providing a far more accurate representation of data similarity than standard Euclidean distance. Instead of relying on an arbitrary coordinate system, the latent space inherits the intrinsic geometry of the data itself. Samuel Gruffaz et al. note that this pullback approach allows neural networks to implicitly define complex manifold structures through their learned transformations, ensuring that latent distances reflect meaningful structural differences in the original input domain, a property also utilized by Luca Ambrogioni et al..
Navigating the Manifold via Geodesics
Once the latent space is equipped with a Riemannian metric, the optimal path between two points is no longer a straight line, but a geodesic—the shortest path along a curved surface.
Euclidean Latent Space (Flat) Riemannian Latent Space (Curved Manifold)
z_0 --------- z_1 z_0 __
\ / \__ (Geodesic Path)
\ / \__
Low-Density z_1
Dead Zone (Stays in High Density)
Geodesics naturally avoid regions of high magnification where the decoder drastically stretches the latent space. Because these high-magnification zones typically correspond to low-density areas where the model lacks confidence, staying clear of them prevents generation artifacts.
The practical challenge is that finding exact geodesics requires solving a complex system of ordinary differential equations (ODEs), which can be computationally prohibitive. Researchers have developed several clever approximations to make this pathfinding efficient:
- Uncertainty Mapping: Clément Chadebec discovered that the variance produced by a VAE's encoder functions as a natural map of the data's geometry. High variance signifies distance from the data manifold, acting as a geometric barrier. Calculating paths that avoid these high-variance regions yields smooth, realistic morphing without explicit Jacobian computation.
- Graph-Based Search: Rather than solving continuous differential equations, Francesco Ferroni et al. propose routing interpolations through a graph constructed over the model's aggregate posterior. By using an A* search to connect existing encoded data points, they find quasi-geodesics in milliseconds.
- Energy-Minimizing Splines: Mike Yan Michelis et al. optimize continuous cubic B-splines to minimize path energy across the manifold. They show that paths following this natural geometry produce intermediate samples that are significantly more distinct and recognizable than those generated by linear interpolation.
Density-Weighted Manifolds and Fermat's Principle
While the pullback metric captures how the generator stretches space, it does not inherently know where the training data is most concentrated. To ensure that interpolations are not just mathematically smooth but also statistically plausible, researchers incorporate data density directly into the Riemannian metric.
This technique draws inspiration from optics. According to Fermat's Principle, light traveling through a medium of varying density bends to take the path of least time. The authors of Weighted Geodesic Distance Following Fermat's Principle treat data density as a "refractive index" for pathfinding. By scaling the metric tensor such that low-density regions are computationally expensive to traverse, optimal trajectories are forced to bend toward dense regions of the feature space.
Ricardo Dominguez-Olmedo et al. leverage this density-weighted framework to generate realistic counterfactual explanations, ensuring that transitions between data states prioritize regions with high statistical support.
To bypass the intense computational overhead of calculating Jacobians for density scaling, Georgios Arvanitidis et al. introduce a simplified alternative: defining a Riemannian metric that is directly inversely proportional to the probability density of a learned latent prior. This elegant formulation ensures that the shortest paths in the latent space naturally gravitate toward high-density data regions, providing a fast track to semantically consistent, manifold-constrained interpolation.