Preserving Geometric Integrity in Non-Euclidean Neural Networks

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2242 papers300 conceptsfrozen 2026-10-04

Most deep learning models are "flat-earthers." They assume that data—whether it is an image, a sentence, or a user profile—exists in a flat Euclidean space where the shortest path between two points is a straight line. However, complex data often lives on curved surfaces called manifolds. Hierarchies, like social networks or biological taxonomies, naturally fit into the expanding volume of hyperbolic space. Rotations in robotics live on Lie groups. Brain connectivity data lives on the manifold of Symmetric Positive Definite (SPD) matrices.

The central challenge in this research direction is moving beyond "pseudo-geometric" models that merely project curved data onto flat planes to do their math. Instead, researchers are building architectures that operate intrinsically on the manifold, preserving the geometric integrity of the data.

The Tangent Space Trap

To use standard neural network tools like matrix multiplication or convolution, many researchers traditionally project manifold data onto a tangent space—a flat Euclidean approximation at a specific point.

As Yan Yang, Mehrtash Harandi, and Conrad Sanderson point out in Multi-shot person re-identification via relational Stein divergence, this "flattening" introduces significant geometric distortion. A tangent space is only accurate near the point of tangency; as data points move further away, the distances calculated in the flat space stop matching the true distances on the curved manifold.

This distortion isn't just a mathematical nuisance—it destroys the model's performance. In Relational divergence based classification on Riemannian manifolds, Harandi demonstrates that these projections compromise the global relationships and structural integrity of the data. Similarly, Zhiwu Huang argues in Deep Learning on Lie Groups for Skeleton-based Action Recognition that mapping skeletal rotations to flat spaces results in a massive loss of geometric information, limiting the effectiveness of standard deep learning.

Staying on the Manifold

To solve this, a new generation of models performs operations directly on the manifold. This is often referred to as "intrinsic" computation.

In the realm of recommendation systems, Bosong Huang and colleagues introduced the Lorentz Equivariant Model for Collaborative Filtering (LECF). By enforcing Lorentz equivariance, they ensure that features remain consistent under geometric transformations without ever needing to project them into a flat space. Similarly, Naicheng Guo and QIONGXU MA developed PHGR, which performs graph operations and inner products directly within the Poincaré ball to capture the power-law nature of user interactions.

One of the most difficult operations to perform intrinsically is the "average." In Euclidean space, you just sum vectors and divide. On a manifold, you need the Fréchet mean. Aaron Lou and Isay Katsman provided a breakthrough in Differentiating through the Fréchet Mean, creating a differentiable framework that allows hyperbolic graph networks to perform batch normalization and aggregation without falling back into the "tangent space trap."

Dimensionality Reduction Without Distortion

Standard techniques like Principal Component Analysis (PCA) assume a flat world. When applied to curved data, they fail to see the hierarchy. Ines Chami and Albert Gu addressed this with HoroPCA, which uses "horospherical projections" instead of linear ones. This preserves the hierarchical structure of hyperbolic data far more effectively than Euclidean PCA.

Taking this further, Xiran Fan and Baba C. Vemuri introduced Nested Hyperbolic Spaces. Instead of projecting a high-dimensional hyperbolic manifold onto a flat space, they project it onto a lower-dimensional hyperbolic manifold. This "manifold-to-manifold" approach, also explored for SPD matrices by Mathieu Salzmann and Mehrtash Harandi in From Manifold to Manifold, ensures that the discriminative structural information is never "flattened" out of existence.

The Cost of Precision

If intrinsic operations are so much better, why aren't they the default? The answer is computational complexity. Hyperbolic and Riemannian functions are often slow and difficult to scale on modern GPUs designed for linear algebra.

Researchers like Nurendra Choudhary and Chandan K. Reddy are bridging this gap. In Towards Scalable Hyperbolic Neural Networks using Taylor Series Approximations, they use polynomial expansions to approximate expensive hyperbolic functions, reducing latency by 85% while avoiding the massive errors introduced by simple tangent space projections.

As Mario Lezcano Casado notes in Trivializations for Gradient-Based Optimization on Manifolds, the goal is to find a middle ground: mathematical frameworks that allow us to use the efficiency of Euclidean optimization while respecting the "corridor" of the manifold's true geometry.

Papers in this version

Showing 60 of 242 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 118 papers
  2. Version 1 · 2026-10-04 · 242 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_tangent_space_approximation_error_be4a487a,
  title = {tangent space approximation error},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tangent-space-approximation-error/rdr_be4a487ae5e14e9e94f35fceaf840f32},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_preserving_geometric_integrity_in_non_eu_be4a487a_v1,
  title = {Preserving Geometric Integrity in Non-Euclidean Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preserving-geometric-integrity-in-non-euclidean-neural-networks/rdr_be4a487ae5e14e9e94f35fceaf840f32/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}