Deep Learning on Hyperbolic Manifolds

This direction is merged into another direction: hierarchical embedding distortion. Its last version stays readable and citable here.

Version 1 of 115 papers19 conceptsfrozen 2026-10-04

Most modern neural networks operate in Euclidean space—the "flat" geometry we learn in school. However, many real-world datasets, such as knowledge graphs, citation networks, and phylogenetic trees, are inherently hierarchical. As Wentao Shi and colleagues point out in FFHR, trying to force these branching structures into flat space is like trying to flatten a globe onto a paper map: it creates massive distortion unless you use an impractically high number of dimensions.

Hyperbolic geometry offers a solution. Because hyperbolic space expands exponentially rather than polynomially, it has "more room" at its edges, making it the natural mathematical home for trees and hierarchies. Recent research is focused on moving beyond simple embeddings to build entire neural architectures—Transformers, CNNs, and GNNs—that live entirely within these curved manifolds.

The Geometry of Hierarchy

In a tree, the number of nodes grows exponentially with depth. In Euclidean space, the volume of a sphere only grows polynomially with its radius ($r^n$). This mismatch means that as a tree grows, Euclidean embeddings must squeeze nodes closer and closer together, losing the structural relationships.

Hyperbolic space solves this because its volume grows exponentially with the radius. This allows hierarchical data to be embedded with nearly zero distortion in very low dimensions. However, as Andrii Skliar notes in Hyperbolic Convolutional Neural Networks, most deep learning operations (like convolutions and message passing) are designed for flat space and do not work out-of-the-box in curved geometry.

Overcoming the Boundary Problem

One of the primary hurdles in Hyperbolic Neural Networks (HNNs) is the "boundary problem." In models like the Poincaré ball, as data points move toward the edge of the space to represent deeper levels of a hierarchy, numerical precision begins to collapse. Ziheng Chen and Bernhard Schölkopf identify this in Proper Velocity Neural Networks, noting that gradients often vanish near these boundaries.

To solve this, researchers are moving toward "unconstrained" representations. Instead of forcing the computer to handle the delicate math of the manifold boundary, they map hyperbolic points to a space where standard deep learning frameworks can operate stably without losing the underlying geometric properties.

Building the Hyperbolic Toolbox

To make HNNs as powerful as their Euclidean counterparts, researchers are rebuilding the standard deep learning "toolbox" from the ground up:

  1. Normalization: Standard Batch Norm relies on Euclidean means. Mehrdad Khatir and Chandan K. Reddy explore using Fréchet means—a way to calculate averages on curved surfaces—to stabilize training in the Pseudo-Poincaré framework.
  2. Linear Layers: Traditional linear layers don't respect the curvature of the manifold. Robert van der Klis and Thomas Hofmann introduced "geometrically grounded" layers in the Lorentz model, which use hyperbolic hyperplanes to perform transformations. This fixed a "pathology" where previous models took an impossible number of steps to learn simple hierarchies.
  3. Attention Mechanisms: The Hypformer by Menglin Yang and Ce Zhang represents the first comprehensive Transformer designed for hyperbolic space. By implementing hyperbolic linear self-attention and native normalization, they achieved a 10x reduction in memory costs when processing massive, billion-node graphs.

Efficiency through Unification

While pure hyperbolic models are mathematically elegant, they are often slower than Euclidean ones. The Pseudo-Poincaré framework suggests a hybrid path: performing the bulk of the computation in a standard Euclidean Graph Neural Network and then applying a specialized "hyperbolic normalization" layer. This approach captures the benefits of hyperbolic geometry—specifically the ability to represent hierarchy—while maintaining the speed and stability of traditional deep learning.

By bridging the gap between curved-space theory and practical engineering, this research direction is making it possible to process the world's most complex, branching datasets with the same efficiency we currently apply to images and text.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_deep_learning_on_hyperbolic_manifolds_ec619fe6,
  title = {Deep Learning on Hyperbolic Manifolds},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/deep-learning-on-hyperbolic-manifolds/rdr_ec619fe6e30f4e538a85a17812d58366},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_deep_learning_on_hyperbolic_manifolds_ec619fe6_v1,
  title = {Deep Learning on Hyperbolic Manifolds},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/deep-learning-on-hyperbolic-manifolds/rdr_ec619fe6e30f4e538a85a17812d58366/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}