Adapting Manifold Curvature for Hierarchical Representation Learning

This direction is merged into another direction: hyperbolic embedding. Its last version stays readable and citable here.

Version 1 of 127 papers29 conceptsfrozen 2026-10-04

Most machine learning models assume that data lives in a "flat" Euclidean world. In this world, the shortest distance between two points is a straight line, and the volume of a sphere grows predictably as its radius increases. However, real-world data—ranging from the taxonomies of species to the structure of social networks and the semantic relationships in language—is often deeply hierarchical.

When we try to squeeze a tree-like structure into a flat space, we encounter a fundamental geometric mismatch. In a tree, the number of nodes grows exponentially with depth. In Euclidean space, the volume only grows polynomially. This leads to "crowding," where nodes at the edges of the tree are forced too close together, destroying the very relationships the model is trying to learn.

To solve this, researchers are moving toward Hyperbolic Space, a geometry with constant negative curvature that naturally expands to accommodate exponential growth.

The Intuition of Negative Curvature

Think of hyperbolic space not as a flat sheet of paper, but as a surface that "ruffles" or "flares" outward, like a head of kale or a coral reef. Because the space itself expands, it can embed a tree with almost zero distortion.

As Valentin Khrulkov, Victor S. Lempitsky, and their colleagues demonstrated in Hyperbolic Image Embeddings, this geometry is particularly powerful for few-shot learning and image retrieval. By mapping images into a Poincaré ball—a common model of hyperbolic space—the model can naturally represent the "is-a" relationships (e.g., Husky is a Dog is an Animal) that are often hidden in visual data.

The Curvature Bottleneck

While the benefits of hyperbolic space are clear, there is a catch: the curvature parameter. This value determines exactly how much the space "bends." If the curvature is too low, the space is too flat to fit the hierarchy; if it is too high, the model becomes overly sensitive to small changes in the data.

Historically, this parameter was treated as a fixed hyperparameter, requiring tedious manual tuning for every new dataset. As noted in Hyperbolic Category Discovery by Yuanpei Liu and Kai Han, different tasks—such as identifying fine-grained vs. generic categories—require different geometric settings. This sensitivity has been a major barrier to the widespread adoption of hyperbolic models.

From Manual Tuning to Adaptive Geometry

The most recent wave of research focuses on making these spaces adaptive. Instead of picking a curvature and hoping for the best, new frameworks learn the optimal geometry alongside the data.

In ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network, Xingcheng Fu and Jianxin Li introduced a reinforcement learning approach where agents "explore" different curvatures to find the best fit for a specific graph's topology. This allows the model to adjust its "bend" based on how tree-like the input data actually is.

Similarly, Zelin Peng and Wei Shen developed HyperET, which uses Möbius multiplication to dynamically adjust the "hyperbolic radius" of features. This allows Multi-modal Large Language Models to align visual and textual hierarchies more efficiently, using less than 1% of the parameters required by traditional methods.

Simplifying the Hyperbolic Pipeline

One reason hyperbolic geometry hasn't replaced Euclidean space entirely is its mathematical complexity. Operations like addition and multiplication don't work the same way in curved space; they require specialized "gyrovector" calculus.

To lower this barrier, Mehrdad Khatir and Chandan K. Reddy proposed the Pseudo-Poincaré framework. They proved that you can often achieve the benefits of hyperbolic geometry by training a standard Euclidean model and applying a specialized "hyperbolic normalization" layer at the end. This approach is up to 5x faster while maintaining the structural advantages of curved embeddings.

A Universal Tool for Structure

The shift toward adaptive hyperbolic embeddings is now reaching diverse fields:

By moving beyond the "flat-earth" assumption of standard neural networks, these researchers are building models that finally match the inherent geometry of the information they process.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_adapting_manifold_curvature_for_hierarch_dc17081d,
  title = {Adapting Manifold Curvature for Hierarchical Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adapting-manifold-curvature-for-hierarchical-representation-learning/rdr_dc17081d96e74afa97d91fd7cee6acae},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_adapting_manifold_curvature_for_hierarch_dc17081d_v1,
  title = {Adapting Manifold Curvature for Hierarchical Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adapting-manifold-curvature-for-hierarchical-representation-learning/rdr_dc17081d96e74afa97d91fd7cee6acae/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}