Deep Learning in Hyperbolic Space

This direction is merged into another direction: hyperbolic embedding, gyrovector space operations. Its last version stays readable and citable here.

Version 1 of 176 papers90 conceptsfrozen 2026-10-04

Most deep learning architectures, from ResNets to Transformers, operate under the assumption that the world is "flat." They embed data into Euclidean space, where the shortest path between two points is a straight line and the volume of a sphere grows polynomially with its radius. However, as Wei Peng and colleagues note in Hyperbolic Deep Neural Networks: A Survey, real-world data is rarely flat. It is filled with hierarchies: taxonomies, part-whole relationships, and scale-free networks.

Hyperbolic geometry offers a mathematically elegant solution to this "flatness" bottleneck by providing a space that is naturally "tree-like."

The Geometry of Hierarchy

To understand why hyperbolic space works, consider a simple tree structure where every node has two children. As you move down the levels, the number of nodes grows exponentially ($2^n$). If you try to embed this tree into a 2D Euclidean plane, you quickly run out of room; the nodes at the bottom of the tree will be crushed together.

In contrast, hyperbolic space has negative curvature. This curvature causes the space to expand exponentially as you move away from the center. In the Poincaré ball model—a common representation of hyperbolic space—the "distance" between points increases as you approach the boundary. This provides the "room" necessary to embed complex hierarchies with very low distortion. As Valentin Khrulkov and Victor S. Lempitsky demonstrate in Hyperbolic Image Embeddings, this geometry allows models to capture generic-to-specific features (like "animal" vs "husky") far more efficiently than Euclidean models.

Doing Math on a Curve

Standard neural network operations—like adding a bias or multiplying by a weight matrix—are fundamentally Euclidean. In a curved manifold, these operations are undefined. To build Hyperbolic Neural Networks (HNNs), researchers like Octavian-Eugen Ganea and Gary Bécigneul introduced a framework based on Möbius gyrovector algebra in their seminal paper Hyperbolic Neural Networks.

The core idea is to replace standard arithmetic with "gyro-operations":

  • Möbius Addition: A way to "translate" points within the Poincaré ball while ensuring they never leave the boundary.
  • Möbius Scalar Multiplication: A way to scale vectors that respects the manifold's curvature.
  • Exponential and Logarithmic Maps: These act as bridges, projecting "flat" Euclidean features into the hyperbolic manifold and back again.

By redefining layers using these operations, we can build hyperbolic versions of multinomial logistic regression, GRUs, and feed-forward networks that remain "native" to the curved space.

Choosing the Right Model

While the Poincaré ball is intuitive for visualization, it can suffer from numerical instability near the boundaries. Weize Chen and Xu Han argue in Fully Hyperbolic Neural Networks that the Lorentz (or Minkowski) model is often superior for deep learning.

The Lorentz model represents hyperbolic space as the surface of a hyperboloid in a higher-dimensional Minkowski space. Instead of Möbius algebra, it uses Lorentz transformations (rotations and boosts). This approach avoids the "boundary problem" where gradients vanish as points approach the edge of the Poincaré disk, leading to more stable training for large-scale tasks like knowledge graph completion.

Applications Across Modalities

The shift to hyperbolic geometry is yielding gains across diverse fields:

Why It Works: The Intuition

Hyperbolic deep learning works because it aligns the inductive bias of the model's geometry with the intrinsic structure of the data. If your data has a hierarchy, forcing it into a Euclidean space is like trying to flatten a globe onto a paper map—you will inevitably get distortion. By moving to a space with negative curvature, we allow the model to represent "depth" and "abstraction" as geometric distance from the origin, creating a more efficient and expressive representation of the world's complexity.

Papers in this version

Showing 60 of 76 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_deep_learning_in_hyperbolic_space_93d84dfb,
  title = {Deep Learning in Hyperbolic Space},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/deep-learning-in-hyperbolic-space/rdr_93d84dfb79394789bfc10b92638b5f3e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_deep_learning_in_hyperbolic_space_93d84dfb_v1,
  title = {Deep Learning in Hyperbolic Space},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/deep-learning-in-hyperbolic-space/rdr_93d84dfb79394789bfc10b92638b5f3e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}