Learning Representations in Mixed Curvature Product Spaces

This direction is merged into another direction: hierarchical embedding distortion. Its last version stays readable and citable here.

Version 1 of 172 papers79 conceptsfrozen 2026-10-04

In representation learning, we usually assume the world is flat. We map complex data—images, words, or social networks—into high-dimensional Euclidean space ($\mathbb{R}^n$). But as the field has matured, researchers have realized that Euclidean space is often the wrong "shape" for the data we want to model.

If you try to embed a tree-like hierarchy into a flat plane, you encounter a fundamental geometric conflict: the number of nodes in a tree grows exponentially with depth, but the volume of Euclidean space only grows polynomially with radius. To fit the tree without massive distortion, you would need an infinite number of dimensions.

The Geometry of Information

To solve this, researchers like Octavian-Eugen Ganea and Gary Bécigneul introduced Hyperbolic Neural Networks. Hyperbolic space has "negative curvature," meaning it expands exponentially as you move away from the center. It is the natural home for hierarchies, taxonomies, and power-law graphs because it provides the "room" that trees need to breathe.

Conversely, some data is cyclical or clustered, like the seasonal patterns in a time series or the structure of a molecule. These patterns are better suited for "positive curvature" spaces, like the surface of a sphere. As Jie Hong et al. argue in Curved Geometric Networks for Visual Anomaly Recognition, using the right curvature is essential for identifying anomalies that don't fit the expected structural "shape" of the data.

The Problem of Topological Heterogeneity

The challenge is that real-world data is rarely "pure." A social network might have a hierarchical backbone (celebrities and followers) but also contains dense, cyclical cliques (friend groups). A knowledge graph contains "is-a" hierarchies alongside symmetric, cyclical relations.

As Menglin Yang and Min Zhou point out in their survey of hyperbolic graph learning, forcing these diverse structures into a single constant-curvature manifold—whether it's flat, spherical, or hyperbolic—causes significant representation inaccuracies. If you optimize for the tree, you distort the cycles; if you optimize for the cycles, you squash the hierarchy.

Mixing Curvatures via Product Manifolds

The emerging solution is to move away from single-geometry manifolds toward Mixed-Curvature Product Spaces. Instead of choosing one geometry, we represent a data point as a combination of coordinates from different spaces:

$$\mathcal{M} = \mathbb{E}^{n_e} \times \mathbb{H}^{n_h} \times \mathbb{S}^{n_s}$$

In this framework, a single embedding vector might have some dimensions living in Euclidean space ($\mathbb{E}$), some in Hyperbolic space ($\mathbb{H}$), and some in Spherical space ($\mathbb{S}$).

Ondrej Skopek and colleagues pioneered this with Mixed-curvature Variational Autoencoders. Their model doesn't just use a mixed space; it learns the optimal curvature for each dimension of the latent space. This allows the model to adaptively allocate "hierarchical dimensions" or "cyclical dimensions" based on the specific needs of the dataset.

Adaptive and Pseudo-Riemannian Approaches

Recent work has focused on making these geometries even more flexible. Xingcheng Fu et al. developed ACE-HGNN, which uses reinforcement learning to explore and select the best curvature for different regions of a graph. Similarly, Li Sun et al. introduced SELFMGNN, a self-supervised framework that captures mixed structural motifs (like trees and cliques) without requiring manual labels.

Some researchers are pushing even further into exotic mathematics. Bo Xiong has explored Ultrahyperbolic Knowledge Graph Embeddings and Pseudo-Riemannian Graph Convolutional Networks. These models use manifolds with "indefinite metrics," which can mathematically interleave hyperbolic and spherical properties within a single unified space, rather than just concatenating them as products.

Why It Works: The Intuition of Alignment

The success of these methods boils down to inductive bias. By matching the mathematical properties of the embedding space to the topological properties of the data, we reduce the "energy" required to learn a representation.

When Menglin Yang uses discrete graph curvature to guide message passing in $\kappa$HGCN, the model isn't just memorizing features; it's aligning the flow of information with the underlying shape of the network. This alignment leads to better generalization, lower dimensionality, and more robust performance on tasks ranging from knowledge graph completion to social text classification.

Papers in this version

Showing 60 of 72 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learning_representations_in_mixed_curvat_adc75c87,
  title = {Learning Representations in Mixed Curvature Product Spaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-representations-in-mixed-curvature-product-spaces/rdr_adc75c87ca594879a0f7caca841076c3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_learning_representations_in_mixed_curvat_adc75c87_v1,
  title = {Learning Representations in Mixed Curvature Product Spaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-representations-in-mixed-curvature-product-spaces/rdr_adc75c87ca594879a0f7caca841076c3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}