Overcoming Euclidean Distortion with Hyperbolic Embeddings

1,359 concepts852 papers13 proposals

When we represent data as vectors in a standard neural network, we almost always default to Euclidean space. We visualize our embeddings as points on a flat plane or in a high-dimensional cube where the shortest path between two points is a straight line.

This flat geometry works remarkably well for continuous, grid-like data, but it hits a mathematical brick wall when confronted with hierarchies. Taxonomies, evolutionary trees, social networks, and semantic ontologies all share a common property: they branch out exponentially. Forcing these branching structures into a flat Euclidean space introduces severe representation distortion, crowding distinct concepts together and stripping the embedding space of its discriminative power.

To understand why this happens—and how to fix it—we have to look at how space expands.

The Volume Mismatch Problem

Imagine trying to draw a perfectly balanced family tree on a flat sheet of paper. The root node has two children, each child has two children, and so on. By the tenth generation, you have over a thousand leaf nodes. If you try to keep the branch lengths (the distances between parents and children) uniform, you will quickly run out of physical space. The leaves will begin to overlap, and the lines will cross.

This is not just a drawing problem; it is a fundamental limitation of Euclidean geometry. In a $d$-dimensional Euclidean space, the volume of a sphere grows polynomially relative to its radius $r$:

$$\text{Volume}_{\text{Euclidean}}(r) \propto r^d$$

However, the number of nodes in a tree or hierarchical taxonomy grows exponentially with its depth $h$, governed by its branching factor $b$:

$$\text{Nodes}(h) \propto b^h$$

As noted by Ganea et al. in their foundational work on Hyperbolic Neural Networks, this creates a geometric mismatch. If you map an exponentially expanding tree into a space that only expands polynomially, you are forced into a compromise. To prevent deep nodes from collapsing into the same coordinates, standard architectures must scale up the dimensionality $d$ excessively. This high-dimensional escape route is computationally expensive and fails to capture the underlying anatomy of the hierarchy in a compact representation.

The Hyperbolic Solution

Hyperbolic geometry solves this bottleneck by introducing constant negative curvature. In a hyperbolic space—such as the Poincaré ball or the Lorentz model—the volume of a sphere grows exponentially with its radius:

$$\text{Volume}_{\text{Hyperbolic}}(r) \propto e^r$$

This exponential growth perfectly mirrors the expansion of a tree. In essence, hyperbolic space has "more room" at its periphery.

Euclidean Space:   Growth is Polynomial O(r^d)  -> Crowding at the edges
Hyperbolic Space:  Growth is Exponential O(e^r) -> Perfect fit for trees

Consider the Poincaré disk model, where the entire infinite hyperbolic space is mapped inside a unit disk. As you move outward from the origin toward the boundary (where the radius approaches 1), the metric tensor scales up. A tiny Euclidean distance near the edge of the disk corresponds to an immense hyperbolic distance.

This property allows an infinite tree to be embedded with arbitrarily low distance distortion. The root of the hierarchy sits naturally at the origin, while the specific leaf nodes spread out toward the boundary, remaining widely separated from one another while maintaining their precise ancestral distances to the root.

Encoding Abstraction into Geometry

By moving away from flat vector spaces, models can encode semantic abstraction directly into the coordinate system. The distance of a point from the origin naturally represents its hierarchical depth or specificity, while its angular direction distinguishes it from its "sibling" concepts.

This geometric intuition allows models to handle uncertainty gracefully. For instance, in Learning the Predictability of the Future, Coll-Vinent et al. leverage hyperbolic properties to predict video actions. When a model is uncertain about the exact future action, it "hedges its bets" by placing its prediction close to the origin—representing a broad parent category like "greeting." When the model gains confidence, the embedding moves outward toward a specific leaf node, such as "handshake."

To make these representations actionable, researchers have had to rebuild deep learning operations from scratch. Because standard matrix multiplications break down in curved spaces, Ganea et al. derived hyperbolic variants of multinomial logistic regression and feed-forward layers using Möbius gyrovector spaces. This framework was later optimized for parameter efficiency by Shimizu et al. in Hyperbolic Neural Networks++, making hyperbolic layers drop-in replacements for standard neural network components.

Applications Across Domains

The mitigation of Euclidean representation distortion has driven performance breakthroughs across several distinct domains:

Beyond Constant Curvature

While hyperbolic space perfectly accommodates pure trees, real-world datasets are rarely uniform; they are often a messy combination of hierarchies, cycles, and flat grids. Forcing a cyclical graph into a hyperbolic space introduces a reverse distortion problem.

To address this, Gu et al. proposed product manifolds in Learning Mixed-Curvature Representations in Product Spaces. By combining spherical spaces (to capture cycles), hyperbolic spaces (to capture hierarchies), and Euclidean spaces (to capture flat structures), they allow models to dynamically learn the optimal mixed-curvature environment for the data.

Alternatively, Cruceru et al. explore non-constant curvature in Computationally Tractable Riemannian Manifolds for Graph Embeddings, utilizing Symmetric Positive Definite (SPD) and Grassmann manifolds to represent complex topologies without forcing them into a single geometric paradigm.

Ultimately, the shift toward hyperbolic and non-Euclidean embeddings represents a fundamental realization in representation learning: the geometry of your embedding space must match the topology of your data. When the data branches, the space must bend.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 921 papers
  2. Version 1 · 2026-10-04 · 1093 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_embedding_distortion_6d3582bf,
  title = {hierarchical embedding distortion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-embedding-distortion/rdr_6d3582bf197e45e4b1181c6d3ccb5bc2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_hierarchical_embedding_distortion_6d3582bf_v2,
  title = {hierarchical embedding distortion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-embedding-distortion/rdr_6d3582bf197e45e4b1181c6d3ccb5bc2/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake