The Limits of Tangent Space Linearization on Curved Manifolds

156 concepts122 papers1 proposal

Many real-world datasets—such as hierarchical trees, graph structures, 3D rotations, and symmetric positive definite (SPD) matrices—live naturally on curved, non-Euclidean manifolds. However, standard machine learning tools, from linear classifiers to deep neural network layers, are fundamentally built for flat Euclidean space.

To bridge this gap, researchers frequently rely on a technique called linearization. This process projects curved manifold data onto a flat, vector-valued tangent space at a specific reference point (or "pole") using a mathematical operation known as the logarithmic map. Once the data is flattened into this tangent space, standard Euclidean algorithms can be applied before the results are projected back onto the manifold via the exponential map.

While computationally convenient, this shortcut introduces a fundamental flaw: tangent space approximation error. Repeatedly forcing curved geometry into a flat plane warps the underlying data structure, leading to a significant loss of geometric information.

The Anatomy of Geometric Distortion

To understand why linearization fails, imagine trying to represent the surface of the Earth on a flat paper map. Near the center of the map, the representation is highly accurate. But as you move toward the edges, greenland stretches to the size of Africa. Tangent space projections suffer from the exact same distortion mechanics.

1. The Distance Decay Effect

A tangent plane is only a perfect local approximation at the exact point of tangency. As data points move further away from this reference center, the flat approximation increasingly warps pairwise distances.

As noted by Oren Freifeld et al. in A Geometric take on Metric Learning, projecting manifold data onto a single tangent space preserves distances relative to the origin but fails to maintain exact pairwise distances between all points across the manifold. Sadeep Jayasumana et al. in Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels emphasize that this geometric distortion worsens significantly for distant points, creating a severe bottleneck for widely distributed data.

2. The Reference Point Bottleneck

Because linearization depends entirely on a specific anchor point (such as the Riemannian Center of Mass), the choice of this reference space dictates the error distribution. If the anchor point is poorly chosen, the resulting representation degrades.

Rushil Anirudh et al. in Elastic Functional Coding of Riemannian Trajectories demonstrate that mapping complex trajectories into a Euclidean tangent space is highly sensitive to this reference selection; an improperly selected anchor introduces massive mathematical distortions and can prevent statistical models from converging. This dependency means that the accuracy of the entire pipeline hinges on a variable that may not reflect the global structure of the data, a challenge also highlighted by Florian Beier et al. in Multi-marginal Approximation of the Linear Gromov-Wasserstein Distance.

        Curved Manifold                     Flat Tangent Space
             _-_                                 ___________
           /     \   --- Logarithmic Map --->   /  .   .   /
          |   .   |                            /  .  X  . /  <- Distances warp
           \_ _ _/   <--- Exponential Map --- /___________/     far from anchor (X)

High Curvature and First-Order Failures

The core assumption of tangent space projection is that the manifold is locally flat. Mathematically, a tangent space acts as a first-order Taylor approximation of the manifold's surface. This approximation breaks down entirely in regions of high curvature or when dealing with non-linear data distributions.

Ilya Kaufman et al. in Curvature Enhanced Data Augmentation for Regression demonstrate that first-order manifold approximations treat local data surfaces as flat planes, causing synthetic points or updates to drift away from the true functional relationship in highly curved regions.

This failure is particularly pronounced in hyperbolic spaces, which are designed to embed hierarchical trees with near-zero distortion because their volume grows exponentially. When hyperbolic data is projected onto a flat tangent plane, the intrinsic hierarchical relationships and exponential distance scaling are completely crushed. As argued by Meimei Yang et al. in Expanding the Hyperbolic Kernels: A Curvature-aware Isometric Embedding View, this projection fails to preserve the very distance relationships and hierarchical benefits that hyperbolic space was chosen to provide in the first place.

Downstream Costs in Machine Learning

When geometric information is lost to linearization, downstream model performance suffers across several paradigms:

Designing Intrinsic Alternatives

To bypass the limits of tangent space approximations, modern architectures increasingly favor fully intrinsic operations. Instead of mapping data back to Euclidean space for computation, these models are designed to perform operations directly on the manifold's native surface.

For example, Qiuyu Liang et al. in Fully Hyperbolic Rotation for Knowledge Graph Embedding and Xiran Fan et al. in Enhancing Hyperbolic Knowledge Graph Embeddings via Lorentz Transformations avoid tangent spaces altogether by defining relational transformations directly on the Lorentz manifold. This preserves structural integrity and eliminates the numerical instability caused by constant re-projection.

Alternatively, when projections are mathematically unavoidable, distortion can be mitigated dynamically. Rama Chellappa et al. in Rolling Rotations for Recognizing Human Actions from 3D Skeletal Data show that by "rolling" a manifold along a moving trajectory rather than relying on a single static anchor point, models can minimize mathematical distortions and preserve geometric integrity across long-range data sequences.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 118 papers
  2. Version 1 · 2026-10-04 · 242 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_tangent_space_approximation_error_be4a487a,
  title = {tangent space approximation error},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tangent-space-approximation-error/rdr_be4a487ae5e14e9e94f35fceaf840f32},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_tangent_space_approximation_error_be4a487a_v2,
  title = {tangent space approximation error},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tangent-space-approximation-error/rdr_be4a487ae5e14e9e94f35fceaf840f32/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake