When we train standard deep learning models, we implicitly assume that our data lives in a flat, Euclidean world. In this world, the shortest distance between two points is a straight line, and the volume of a sphere grows polynomially with its radius.
However, much of the data we care about is intrinsically non-Euclidean. Consider a tree structure, such as a word hierarchy or a social network: the number of nodes grows exponentially with the depth of the tree. If we try to force this exponential structure into a flat Euclidean space, the embeddings suffer from severe distortion. To represent hierarchies faithfully without exploding our dimensionality, we need to move to spaces with negative curvature, such as hyperbolic space, where the volume of a space grows exponentially with its radius.
The challenge is that deep learning is fundamentally built on linear algebra. Operations like layer transformations ($Wx + b$), residual connections ($x + f(x)$), and optimization steps ($x_t - \eta \nabla x_t$) assume the axioms of a standard vector space. If you take two points inside a bounded hyperbolic model—like the Poincaré ball—and perform standard Euclidean addition, the resulting point will often fly right out of the boundary.
To build deep neural networks directly within curved manifolds, we need a new mathematical grammar. Gyrovector space operations provide this exact framework, generalizing standard vector arithmetic to non-Euclidean geometries.
The Problem with Curvature
In a flat vector space, vector addition is commutative ($u + v = v + u$) and associative ($(u + v) + w = u + (v + w)$). These properties seem so foundational that it is hard to imagine arithmetic without them. Yet, the moment a space becomes curved, these properties break down.
Imagine standing on the equator of a globe. If you walk 1,000 miles north and then 1,000 miles east, you end up at a completely different spot than if you had walked 1,000 miles east first and then 1,000 miles north. In curved geometry, the order of translations matters. Geometrically, moving a vector along a curved path alters its orientation—a phenomenon known as parallel transport.
Because standard vector addition does not account for this geometric rotation, it fails to preserve the boundaries and intrinsic distances of curved manifolds. As established by Ganea et al. in their formulation of Hyperbolic Neural Networks (HNNs), we must replace standard vector spaces with gyrovector spaces. A gyrovector space relaxes the rules of associativity and commutativity by introducing an algebraic operator called gyration, which acts as a corrective rotation to account for the space's curvature.
The Mechanics of Möbius Arithmetic
The most common realization of gyrovector space operations in deep learning is Möbius arithmetic, defined over the Poincaré ball. By replacing standard operations with their Möbius counterparts, we can execute the core building blocks of neural networks directly within curved spaces.
Möbius Addition
Instead of shifting a point along a flat grid, Möbius addition ($\oplus_c$, where $c$ denotes the curvature of the space) translates a point along a curved geodesic. Intuitively, as a point moves closer to the boundary of the Poincaré ball, Möbius addition acts like a relativistic speed limit: it decelerates the translation so that the resulting vector asymptotically approaches the edge but never escapes. Balazevic et al. leveraged this operation in their Multi-Relational Poincaré (MuRP) model to translate entity embeddings along relational trajectories, allowing the network to naturally capture complex, overlapping hierarchies in knowledge graphs.
Möbius Scalar Multiplication
We cannot scale a hyperbolic vector by simply multiplying its coordinates by a constant. Instead, Möbius scalar multiplication ($\otimes_c$) scales the distance of a point from the origin along its unique geodesic path. If you multiply a hyperbolic vector by two, it slides twice as far down its curved trajectory away from the origin.
Möbius Linear Layers
A standard neural network layer computes a matrix-vector multiplication ($Wx$). To perform this in a gyrovector space, models use a combination of exponential and logarithmic maps to bridge the curved manifold and its flat tangent space.
To apply a weight matrix $W$ to a hyperbolic vector:
- The vector is projected onto a flat Euclidean tangent space at the origin using a logarithmic map.
- The standard matrix multiplication $W$ is applied in this flat tangent space.
- The resulting vector is projected back into the hyperbolic manifold using an exponential map.
As shown by Li et al., these Möbius linear layers allow traditional neural network backbones to perform differentiable attribute editing directly within a Poincaré disk, where the radius naturally maps to semantic abstraction.
Building Deep Architectures in Curved Spaces
By swapping standard arithmetic for gyrovector operations, researchers have successfully rebuilt the entire deep learning toolkit for non-Euclidean data.
Classification and Decision Boundaries
In Euclidean networks, classification is performed by finding a flat hyperplane that separates classes. In hyperbolic space, a hyperplane becomes a curved decision boundary. Khrulkov et al. adapted multinomial logistic regression (MLR) to the Poincaré ball for computer vision, allowing models to classify images based on hierarchical relationships. Taking this further, Federico Lopez designed a fully hyperbolic model for multi-class classification where every single operation—from initial encoding to final class boundary evaluation—remains strictly within the manifold, preserving structural constraints with a fraction of the parameters required by Euclidean models.
Sequence and Graph Modeling
Temporal and structural dependencies also benefit from gyrovector arithmetic. For sequential data, Sawhney et al. adapted recurrent architectures (like GRUs) to execute internal state updates via Möbius transformations, capturing the scale-free dynamics of financial and social media data. For graph structures, Zhang et al. introduced hyperbolic graph attention mechanisms (HAT). By using the gyrovector framework to compute attention weights and aggregate node features, the network preserves the hierarchical inductive biases of complex graphs across multiple layers.
When data contains a mixture of hierarchical and non-hierarchical structures, operations can even be extended to product spaces. Takeuchi et al. demonstrated that treating weight matrices as collections of sub-blocks allows gyrovector operations to coordinate information flow across a product of multiple distinct hyperbolic spaces simultaneously.
Beyond Hyperbolic Geometry: Matrix Manifolds
The mathematical elegance of gyrovector spaces is that they are not restricted to hyperbolic geometry. Recent work has extended this algebraic rigor to more complex matrix manifolds, such as Symmetric Positive Definite (SPD) matrices and Grassmann manifolds.
Xuan Son Nguyen demonstrated that gyrovector structures can be derived for non-negatively curved spaces and matrix manifolds. By mapping geometric concepts like parallel transport directly to algebraic gyration, Xuan Son Nguyen defined closed-form, manifold-aware versions of addition and scalar multiplication that allow recurrent neural networks to process sequences of covariance or Symmetric Positive Semi-definite matrices without breaking their structural constraints.
Building on this foundation, Nguyen et al. generalized missing geometric concepts like inner products, gyroangles, and hyperplanes to homogeneous Riemannian manifolds. This algebraic extension was fully realized in Matrix Manifold Neural Networks++ by Nguyen et al., who implemented convolutional and fully-connected layers directly on SPD and Grassmann manifolds.
Ultimately, gyrovector space operations provide a unified mathematical bridge. They allow deep learning architectures to step away from the limitation of flat vector spaces, ensuring that no matter how curved or constrained the underlying data geometry is, our networks can calculate, learn, and optimize within it natively.
Key Foundational Work
Hyperbolic Neural Networks (2018) — contributed 1 concept elements to this direction.
Multi-relational Poincaré Graph Embeddings (2019) — contributed 1 concept elements to this direction.