Automated Discovery of Latent Geometric Manifolds

41 concepts35 papers1 proposal

For years, machine learning operated under a flat-Earth assumption. Standard neural networks default to embedding data into Euclidean space—a perfectly flat, familiar coordinate system where the shortest distance between two points is always a straight line.

However, real-world data is rarely flat. Complex relational structures, like phylogenetic trees or corporate hierarchies, expand exponentially as you move down their branches. Forcing this hierarchical data into a flat Euclidean space causes severe geometric distortion, crowding the leaf nodes together and destroying their structural relationships. To solve this, researchers turned to non-Euclidean geometries: hyperbolic spaces (which possess negative curvature and naturally accommodate tree-like hierarchies) and spherical spaces (which possess positive curvature and perfectly model cyclical or bounded data).

As shown by Weber in Exploiting Data Geometry in Machine Learning, identifying and leveraging these intrinsic geometries can transform complex, non-convex optimization problems into highly manageable ones. Yet, this geometric revolution introduced a glaring bottleneck: How do we know which manifold to choose?

Historically, selecting a manifold has been a manual guessing game. Practitioners had to rely on domain expertise or trial-and-error heuristics to pre-select a geometric prior. If the guess was wrong, the model's performance suffered. The emerging field of automated manifold discovery aims to eliminate this human intervention entirely, developing algorithms that can autonomously infer, select, and optimize the geometric shape of a latent space directly from raw data.


The Combinatorial Nightmare of Product Spaces

The challenge intensifies when dealing with heterogeneous data. A real-world graph—such as a social network—is rarely just a hierarchy or just a cycle; it is often a messy combination of both, containing dense hierarchical communities connected by cyclical loops.

To represent this complexity, researchers developed product manifolds. A product manifold stitches together multiple distinct factor spaces into a single embedding environment—for example, combining two hyperbolic spaces, a sphere, and a flat Euclidean plane ($\mathbb{H}^2 \times \mathbb{H}^2 \times \mathbb{S}^3 \times \mathbb{R}^1$).

                       ┌─── Hyperbolic Space (Hierarchies)
                       ├─── Spherical Space  (Cycles)
Raw Heterogeneous ────┼─── Euclidean Space  (Flat features)
     Data              │
                       └─── Automated Selection ───► Optimal Product Manifold

The specific combination of curvatures and dimensionalities allocated to these factor spaces is known as the geometric signature. As Gu et al. demonstrated in Learning Mixed-Curvature Representations in Product Spaces, matching the signature to the data yields massive gains in representation fidelity.

However, finding this signature manually is a combinatorial nightmare. If you have a library of three geometry types (Euclidean, spherical, hyperbolic) and want to allocate them across a 10-dimensional latent space, the number of discrete configurations is immense. As noted by Skopek et al. in Mixed-curvature Variational Autoencoders, this combinatorial explosion forces practitioners to rely on computationally ruinous grid searches.

This manual dependency limits adaptability across diverse domains. Whether trying to classify image sets on Riemannian manifolds (Wang et al.), embed knowledge graphs (Xu et al.), or infer latent graphs (Sáez de Ocáriz Borde et al.; Arroyo et al.), models remain bottlenecked by our inability to optimize the underlying geometry alongside the model's primary weights.


How Automated Selection Works

To move past manual heuristics, recent frameworks treat the selection of geometry either as an informed discrete search or as a continuous optimization problem.

1. Informed Search via Gromov-Hausdorff Distance

One elegant way to automate this process is to measure the "topological distance" between the empirical data and a candidate manifold. In Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization, Sáez de Ocáriz Borde et al. introduced a principled framework (NLGS) that avoids brute-force grid searching.

Instead of training a new model for every possible signature, NLGS uses the Gromov-Hausdorff (GH) distance. The GH distance mathematically computes how far apart two metric spaces are from being perfectly isometric (distance-preserving). By calculating the GH distance between the pairwise distances of the raw data and the intrinsic distances of a candidate product manifold, the algorithm obtains a cheap, proxy signal of how well that manifold fits the data. This signal is then used to guide a Bayesian Optimization loop, rapidly converging on the optimal geometric signature without full model training.

2. Trainable and Continuous Geometries

A second paradigm bypasses discrete searching entirely by making the geometry itself continuous and differentiable.

In Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries, Sáez de Ocáriz Borde et al. proposed a framework where the latent geometries are parameterized and trained end-to-end via gradient descent. Instead of picking a rigid manifold type upfront, the model learns a continuous transformation that warps the latent space dynamically.

A similar philosophy is seen in continual learning. When a model encounters a streaming sequence of tasks, the underlying data distribution changes over time. Gao et al. addressed this in Exploring Data Geometry for Continual Learning by designing a system that dynamically expands and mutates a mixed-curvature space. As new hierarchical or cyclical concepts arrive in the data stream, the latent space allocates new geometric dimensions on the fly, preventing catastrophic forgetting by preserving the structural history of the data.


Broad Applications: From Vision to Robotics

Automating the discovery of data geometry is unlocking performance gains across diverse machine learning domains:

  • Computer Vision & Structure-from-Motion: In 3D reconstruction, frameworks often struggle to decide whether to prioritize point-based or line-based geometric primitives for different scene regions. Zhang et al. demonstrate in Ray Space Features for Plenoptic Structure-from-Motion how analyzing ray manifolds can automate pose estimation in complex scenes, removing the need for manual scene priors.
  • Interpretability in LLMs: We often think of Large Language Models as mapping concepts to simple linear directions. However, Modell et al. proved in The Origins of Representation Manifolds in Large Language Models that LLMs represent continuous concepts (like time or color) as complex geometric manifolds. Automated discovery allows us to map these abstract conceptual spaces without manually guessing their metric structure.
  • Robot Dynamics and Control: Traditional geometric control requires engineers to manually specify a robot's state space based on known physical characteristics. In Learning Dynamics under Environmental Constraints via Measurement-Induced Bundle Structures, Zheng et al. introduced a method to automatically learn complex fiber bundle geometries directly from raw, noisy sensor measurements, providing safety guarantees without pre-defined templates.

Conclusion

The shift toward automated manifold selection represents a maturation of geometric machine learning. By treating the curvature, dimensionality, and composition of latent spaces as learnable properties rather than static hyperparameters, these techniques pave the way for true geometry-agnostic AI—systems capable of discovering not just the patterns within data, but the intrinsic shape of the information itself.

Key Foundational Work

Learning Euclidean-to-Riemannian Metric for Point-to-Set Classification (2014) — contributed 2 concept elements to this direction.
Ray Space Features for Plenoptic Structure-from-Motion (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_automated_manifold_selection_863ccc51,
  title = {automated manifold selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-manifold-selection/rdr_863ccc514dd7435785087dc46d366537},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_automated_manifold_selection_863ccc51_v1,
  title = {automated manifold selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-manifold-selection/rdr_863ccc514dd7435785087dc46d366537/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake