Many real-world datasets—such as hierarchical trees, graph structures, 3D rotations, and symmetric positive definite (SPD) matrices—live naturally on curved, non-Euclidean manifolds. However, standard machine learning tools, from linear classifiers to deep neural network layers, are fundamentally built for flat Euclidean space.
To bridge this gap, researchers frequently rely on a technique called linearization. This process projects curved manifold data onto a flat, vector-valued tangent space at a specific reference point (or "pole") using a mathematical operation known as the logarithmic map. Once the data is flattened into this tangent space, standard Euclidean algorithms can be applied before the results are projected back onto the manifold via the exponential map.
While computationally convenient, this shortcut introduces a fundamental flaw: tangent space approximation error. Repeatedly forcing curved geometry into a flat plane warps the underlying data structure, leading to a significant loss of geometric information.
The Anatomy of Geometric Distortion
To understand why linearization fails, imagine trying to represent the surface of the Earth on a flat paper map. Near the center of the map, the representation is highly accurate. But as you move toward the edges, greenland stretches to the size of Africa. Tangent space projections suffer from the exact same distortion mechanics.
1. The Distance Decay Effect
A tangent plane is only a perfect local approximation at the exact point of tangency. As data points move further away from this reference center, the flat approximation increasingly warps pairwise distances.
As noted by Oren Freifeld et al. in A Geometric take on Metric Learning, projecting manifold data onto a single tangent space preserves distances relative to the origin but fails to maintain exact pairwise distances between all points across the manifold. Sadeep Jayasumana et al. in Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels emphasize that this geometric distortion worsens significantly for distant points, creating a severe bottleneck for widely distributed data.
2. The Reference Point Bottleneck
Because linearization depends entirely on a specific anchor point (such as the Riemannian Center of Mass), the choice of this reference space dictates the error distribution. If the anchor point is poorly chosen, the resulting representation degrades.
Rushil Anirudh et al. in Elastic Functional Coding of Riemannian Trajectories demonstrate that mapping complex trajectories into a Euclidean tangent space is highly sensitive to this reference selection; an improperly selected anchor introduces massive mathematical distortions and can prevent statistical models from converging. This dependency means that the accuracy of the entire pipeline hinges on a variable that may not reflect the global structure of the data, a challenge also highlighted by Florian Beier et al. in Multi-marginal Approximation of the Linear Gromov-Wasserstein Distance.
Curved Manifold Flat Tangent Space
_-_ ___________
/ \ --- Logarithmic Map ---> / . . /
| . | / . X . / <- Distances warp
\_ _ _/ <--- Exponential Map --- /___________/ far from anchor (X)
High Curvature and First-Order Failures
The core assumption of tangent space projection is that the manifold is locally flat. Mathematically, a tangent space acts as a first-order Taylor approximation of the manifold's surface. This approximation breaks down entirely in regions of high curvature or when dealing with non-linear data distributions.
Ilya Kaufman et al. in Curvature Enhanced Data Augmentation for Regression demonstrate that first-order manifold approximations treat local data surfaces as flat planes, causing synthetic points or updates to drift away from the true functional relationship in highly curved regions.
This failure is particularly pronounced in hyperbolic spaces, which are designed to embed hierarchical trees with near-zero distortion because their volume grows exponentially. When hyperbolic data is projected onto a flat tangent plane, the intrinsic hierarchical relationships and exponential distance scaling are completely crushed. As argued by Meimei Yang et al. in Expanding the Hyperbolic Kernels: A Curvature-aware Isometric Embedding View, this projection fails to preserve the very distance relationships and hierarchical benefits that hyperbolic space was chosen to provide in the first place.
Downstream Costs in Machine Learning
When geometric information is lost to linearization, downstream model performance suffers across several paradigms:
- Clustering and Classification: In unsupervised learning, flattening the space strips away the structural relationships needed to identify clusters or subspaces. For instance, Ming Yin et al. in Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds show that flattening an SPD manifold into a tangent space results in a significant loss of intrinsic geometric information, reducing clustering accuracy. Similarly, Masoud Faraki et al. in More about VLAD: A leap from Euclidean to Riemannian manifolds find that linearization discards discriminative context, reducing the performance of global visual descriptors.
- Representation Learning: In deep networks, repeatedly bouncing between curved manifolds and flat tangent spaces acts as an informational bottleneck. Naicheng Guo et al. in Poincaré Heterogeneous Graph Neural Networks for Sequential Recommendation observe that transitioning between manifolds introduces cumulative errors that undermine the non-Euclidean inductive bias entirely.
Designing Intrinsic Alternatives
To bypass the limits of tangent space approximations, modern architectures increasingly favor fully intrinsic operations. Instead of mapping data back to Euclidean space for computation, these models are designed to perform operations directly on the manifold's native surface.
For example, Qiuyu Liang et al. in Fully Hyperbolic Rotation for Knowledge Graph Embedding and Xiran Fan et al. in Enhancing Hyperbolic Knowledge Graph Embeddings via Lorentz Transformations avoid tangent spaces altogether by defining relational transformations directly on the Lorentz manifold. This preserves structural integrity and eliminates the numerical instability caused by constant re-projection.
Alternatively, when projections are mathematically unavoidable, distortion can be mitigated dynamically. Rama Chellappa et al. in Rolling Rotations for Recognizing Human Actions from 3D Skeletal Data show that by "rolling" a manifold along a moving trajectory rather than relying on a single static anchor point, models can minimize mathematical distortions and preserve geometric integrity across long-range data sequences.