Learnable Manifold Curvature in Geometric Deep Learning

461 concepts269 papers1 proposal

Standard deep learning architectures operate under a quiet assumption: the latent spaces where representations live are flat. In Euclidean space, the shortest distance between two points is a straight line, and the volume of a sphere grows polynomially with its radius.

However, much of the data we care about—such as social networks, linguistic taxonomies, and molecular graphs—is inherently non-Euclidean. For instance, hierarchical trees expand exponentially as they grow deeper. Forcing an exponentially expanding tree into a flat Euclidean space introduces massive geometric distortion, squashing distinct branches together.

To solve this, researchers transitioned to non-Euclidean geometries. As Ganea demonstrated in Hyperbolic Neural Networks, and Liu et al. extended in Hyperbolic Graph Neural Networks., hyperbolic space naturally expands exponentially, providing the perfect mathematical "room" to embed hierarchical structures without distortion.

Yet, early iterations of these non-Euclidean models introduced a new bottleneck: they relied on a fixed curvature hyperparameter (such as setting the curvature $c = -1$). This rigid approach assumes that every dataset, every layer, and every relationship shares the exact same topological density. The emerging paradigm of learnable manifold curvature addresses this limitation by treating the geometry of the embedding space itself as a dynamic, differentiable parameter.


The Fixed Curvature Bottleneck

Why is a static manifold geometry problematic? Consider two different datasets: a deep, highly branching corporate hierarchy and a shallow, loosely connected web graph. If we embed both into a hyperbolic space with the same fixed negative curvature, the space will over-allocate volume for the shallow graph, causing its representations to collapse into a narrow region.

Furthermore, during the training of a neural network, the internal data distribution shifts as weights are updated. If the underlying geometry is static, the network must continuously force its evolving representations to conform to a rigid space. As Sawhney et al. noted in AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP, this lack of flexibility creates a representation bottleneck, preventing the geometric space from synchronizing with the model's internal data distribution.


Differentiable Curvature as a Geometric Zoom Lens

The core mechanism of learnable curvature is simple yet powerful: treat the curvature parameter ($c$) of a Riemannian manifold as a parameter updated via backpropagation alongside standard neural network weights.

To understand why this works, think of curvature as a geometric zoom lens. As Li et al. explained in Geometry-aware Distance Measure for Diverse Hierarchical Structures in Hyperbolic Spaces, adjusting the curvature changes the resolution of the space:

  • High Negative Curvature: Zooms in on deep hierarchies, providing the necessary mathematical volume and resolution to distinguish between many fine-grained levels.
  • Low/Zero Curvature: Flattens out the space, making it more effective for representing simpler, more linear, or Euclidean-like relationships.
   Fixed Curvature (Rigid Space)            Learnable Curvature (Adaptive Space)

       |   /\   |  (Data must warp              |   /\   |  (Space warps to
       |  /  \  |   to fit the rigid            |  /  \  |   perfectly cradle
       | /____\ |   geometry)                   \______/     the hierarchy)

By allowing the optimization algorithm to adjust this lens, the model minimizes distortion automatically. In fact, Bdeir et al. demonstrated in Robust Hyperbolic Learning with Curvature-Aware Optimization that letting the geometric space evolve alongside feature representations leads to faster convergence and higher accuracy compared to static alternatives.


Granularities of Adaptation

Curvature learning operates across several levels of granularity, depending on how localized the topological variations are:

1. Layer-wise Adaptation

In deep architectures like Transformers, different layers process different levels of abstraction. Yang et al. introduced layer-wise trainable hyperbolic curvature in Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space. This allows the network to dynamically adjust its representational capacity at different stages of processing, using highly curved spaces where hierarchical reasoning is dense and flatter spaces where features are more distributed.

2. Relation-specific Adaptation

Knowledge graphs contain diverse relation types. For example, a has_part relation is strictly hierarchical, whereas a is_married_to relation is symmetric and flat. In Low-Dimensional Hyperbolic Knowledge Graph Embeddings, Chami et al. designed a model that learns a unique curvature parameter for each individual relationship. The model automatically assigns high curvature to tree-like relations while maintaining a flatter, Euclidean-like geometry for non-hierarchical ones.

3. Task and Pair-wise Generation

Instead of learning a static parameter via gradient descent, some frameworks generate curvature on the fly. For few-shot learning, Gao et al. proposed a method in Curvature Generation in Curved Spaces for Few-Shot Learning that dynamically computes task-specific curvature parameters based on the statistical properties of the input data. At an even finer scale, Li et al. utilized a neural module to estimate pair-specific curvatures by analyzing second-order interactions between feature vectors, allowing the geometry to adapt to the exact structural complexity between two individual points.


Mixed-Curvature and Product Manifolds

Real-world datasets are rarely uniform; they are often a complex mixture of hierarchies, cycles, and flat structures. For instance, a network might have a tree-like organizational structure (hyperbolic) but also contain cyclical communication loops (spherical).

To capture these hybrid topologies, Gu et al. introduced product manifolds in Learning Mixed-Curvature Representations in Product Spaces. Instead of picking a single geometry, the embedding space is constructed as a combination of multiple component spaces (e.g., Hyperbolic $\times$ Spherical $\times$ Euclidean).

By treating the curvatures and scaling factors of these individual components as learnable parameters, the model automatically discovers the latent signature of the data. Borde leveraged this in Latent Graph Inference using Product Manifolds to dynamically optimize the geometric prior of the embedding space, while Guo developed a Mixture of Riemannian Experts framework in GraphMoRE to adaptively assign nodes to personalized combinations of curved spaces based on local structural patterns.

Similarly, the authors of Anomaly and Change Detection in Graph Streams through Constant-Curvature Manifold Embeddings demonstrated that optimizing curvature as a continuous parameter reveals that standard Euclidean space often represents a local maximum for distortion. By letting the optimization process navigate between spherical, flat, and hyperbolic regimes, the model naturally settles on the most faithful geometric mapping for the data.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learnable_manifold_curvature_ddc64077,
  title = {learnable manifold curvature},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learnable-manifold-curvature/rdr_ddc6407749964dbe96266e9aecb36383},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_learnable_manifold_curvature_ddc64077_v1,
  title = {learnable manifold curvature},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learnable-manifold-curvature/rdr_ddc6407749964dbe96266e9aecb36383/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake