When we train deep learning models, we almost always default to embedding our data into flat, Euclidean space ($\mathbb{R}^n$). This choice is mathematically convenient, but it forces a severe geometric assumption onto our data: it assumes the underlying relationships are uniform and flat.
Real-world data is rarely uniform. A social network, a molecular graph, or an abstract syntax tree of a computer program contains a messy mixture of structures. It might have strict branching hierarchies (like a family tree), tightly knit cyclical cliques (like a loop of friends), and flat continuous features (like age or physical coordinates). Squeezing these heterogeneous topologies into a single flat space causes high representation distortion.
To solve this, researchers have developed mixed-curvature product spaces. Instead of forcing data into a "one-size-fits-all" geometry, this approach constructs an embedding space from the Cartesian product of multiple distinct geometries—hyperbolic, spherical, and Euclidean. By giving neural networks access to a menu of curvatures, the data can naturally find its optimal geometric home.
The Geometric Mismatch
To understand why mixed curvatures work, we must first look at how different geometries handle space. The core issue is volume growth—how much "room" a space has as you move away from the origin.
- Euclidean Space (Zero Curvature): The volume of a sphere grows polynomially ($r^d$) relative to its radius. This is ideal for continuous, grid-like data.
- Hyperbolic Space (Negative Curvature): The volume grows exponentially ($e^r$). This matches the properties of trees and hierarchies, where the number of nodes branches out exponentially with depth. Squeezing a tree into Euclidean space causes "crowding" near the edges, but hyperbolic space has plenty of room.
- Spherical Space (Positive Curvature): The volume is bounded, and geodesics (shortest paths) wrap back around on themselves. This is perfect for cyclical patterns, periodic loops, or dense, fully connected cliques.
If a dataset contains both hierarchies and cycles, choosing only hyperbolic or only spherical space creates a fundamental conflict. For example, as noted by Roshni Iyer et al. in their work on Dual-Geometric Space Embedding Model for Two-View Knowledge Graphs, knowledge graphs contain both hierarchical "concepts" and cyclical "instance" relationships. Forcing both into a single uniform geometry inevitably warps one of the two structures.
Constructing the Product Manifold
The solution is to build a composite space using the Cartesian product of simpler Riemannian manifolds:
$$\mathcal{M} = \mathcal{H}^{d_1} \times \mathcal{S}^{d_2} \times \mathbb{R}^{d_3}$$
where $\mathcal{H}$ represents hyperbolic spaces, $\mathcal{S}$ represents spherical spaces, and $\mathbb{R}$ represents Euclidean spaces.
In this combined space, an embedding vector is simply concatenated from sub-vectors belonging to each geometry. The global distance between two points decomposes cleanly into the combination of their distances within each component space.
In a seminal paper, Albert Gu et al. introduced this framework in Learning Mixed-Curvature Representations in Product Spaces. They demonstrated that by estimating a dataset's "geometric signature" using local sectional curvatures, models can automatically determine the optimal combination of hyperbolic, spherical, and Euclidean components before training even begins. This mixed-curvature approach yielded massive reductions in representation distortion for complex network graphs.
[ Input Data Component ]
/ | \
/ | \
v v v
Hyperbolic Spherical Euclidean
(Hierarchies) (Cycles) (Flat Features)
\ | /
\ | /
v v v
[ Combined Product Space ]
This multi-faceted representation allows different dimensions to specialize. As shown by Alexandru Tifrea and Gary Bécigneul in Poincare Glove: Hyperbolic Word Embeddings, embedding entities into a product of low-dimensional hyperbolic spaces allows separate dimensions to capture independent hierarchical facets of a single concept—such as tracking an occupation taxonomy in one component and a scientific taxonomy in another.
How Neural Networks Operate Across Curvatures
Operating within a product space requires rethinking standard neural network operations like layers, attention, and splits, which traditionally rely on Euclidean linear algebra.
Parallel Interaction Tracks
One common design pattern is to process data through parallel geometric tracks and allow them to exchange information. In Graph Geometry Interaction Learning, Shichao Zhu et al. developed an architecture that runs Euclidean and hyperbolic embeddings in parallel. An interaction module allows the tracks to "talk" to each other, adaptively weighting whether a specific node benefits more from flat local clustering or non-Euclidean hierarchical branching.
Similarly, for computer vision and video analysis, Jiaxu Leng et al. proposed Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence Detection. Their framework uses Euclidean space to capture raw, local visual features, while concurrently using hyperbolic space to model the complex, hierarchical progression of video events over time.
Mixture of Riemannian Experts
Instead of processing all data through every geometry, more recent architectures use a Mixture of Experts (MoE) design to route data dynamically. In GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts, Zihao Guo et al. assigned different Riemannian manifolds to individual experts. Nodes are adaptively routed to specific geometric experts based on their local structural patterns.
This routing philosophy is echoed in graph prompting frameworks like Dynamic Graph Prompting via Topology-Routed Mixed-Curvature Experts, which demonstrates that the diversity of geometric curvatures among experts is a far more potent driver of performance than simply increasing the raw parameter count of a single-geometry model.
Mixed-Curvature Transformers
To scale this concept to attention-based architectures, Sungjun Cho et al. introduced the Fully Product-Stereographic Transformer in Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning. By treating curvature as a learnable parameter and generalizing attention to non-Euclidean spaces, they enabled individual attention heads to specialize in distinct topologies. One head might compute attention over a hyperbolic subspace to resolve global hierarchy, while a neighboring head focuses on a spherical subspace to resolve local cyclical clusters.
Why It Works: The Power of Inductive Bias
The success of mixed-curvature product spaces comes down to providing the network with the correct inductive bias.
When a model is restricted to Euclidean space, it must expend significant parameter capacity and optimization effort just to approximate non-Euclidean structures (like trying to draw a tree on a flat sheet of paper without lines crossing or distances distorting). By providing a native product manifold, the mathematical properties of the space do the heavy lifting.
As Philippe Chlenski et al. demonstrated in Mixed-curvature decision trees and random forests, allowing decision trees to split data across angles in mixed-curvature subspaces respects the geometry of the data, leading to better generalization across diverse classification benchmarks. Ultimately, matching the curvature of the embedding space to the intrinsic topology of the data minimizes representation distortion, stabilizes training, and allows deep learning models to capture the true shape of complex information.