In flat Euclidean space, optimization and learning algorithms enjoy predictable rules: if you walk in a straight line, you never return to your starting point, and the distance between two parallel paths remains constant. However, many modern machine learning problems deal with data that naturally lives on curved surfaces, or Riemannian manifolds—such as the space of structured covariance matrices in signal processing, or the rotation groups governing robotic joints.
To design algorithms for these curved spaces, researchers rely on Riemannian geometry. Yet, a profound theoretical divide splits this field. While optimization and learning algorithms have achieved remarkable success on manifolds with non-positive curvature (such as hyperbolic spaces and Hadamard manifolds), they frequently hit a wall when applied to spaces with positive curvature (such as spheres).
Understanding why algorithms thrive in non-positive curvature—and why they break down when curvature becomes positive—reveals the fundamental geometric constraints shaping modern optimization theory.
The Sanctuary of Non-Positive Curvature
To understand why non-positive curvature is the "default" setting for many geometric algorithms, we must look at how paths behave in these spaces. On a manifold, the equivalent of a straight line is a geodesic—the shortest path between two points.
In a space with non-positive curvature, geodesics naturally diverge from one another. A crucial consequence of this divergence is that between any two points on the manifold, there is exactly one unique shortest geodesic. This global uniqueness allows the classical notion of convexity to carry over cleanly from Euclidean space. If a function is geodesically convex, any local minimum is guaranteed to be a global minimum.
Because of this property, researchers have successfully ported powerful Euclidean optimization toolkits to non-positively curved Hadamard manifolds:
- Global Acceleration: In Euclidean space, Nesterov’s accelerated gradient descent allows algorithms to converge significantly faster than standard gradient descent. By exploiting the predictable geometry of Hadamard manifolds, David Martínez-Rubio et al. developed frameworks in Accelerated Riemannian Optimization: Handling Constraints to Bound Geometric Penalties and Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penalties that achieve global acceleration while rigorously handling constraints.
- Curvature-Independent Bounds: Typically, optimization on manifolds incurs a "geometric tax"—a penalty in the convergence rate that depends on how curved the space is. However, Emre Sahinoglu et al. demonstrated in Online Optimization on Hadamard Manifolds: Curvature-Independent Regret Bounds on Horospherically Convex Objectives that by utilizing specialized geometric constructs like horospherical convexity, online learning algorithms can achieve performance bounds entirely independent of the manifold's curvature.
- Stochastic Approximation: Standard optimization algorithms like Stochastic Gradient Descent (SGD) also rely on this well-behaved geometry. Alain Durmus et al. established rigorous convergence guarantees for fixed step-size stochastic schemes in On Riemannian Stochastic Approximation Schemes with Fixed Step-Size, a feat made possible by the non-positive sectional curvature constraint.
Why Positive Curvature Breaks Algorithms
When we transition to positively curved spaces like hyperspheres, the geometric rules change completely. Instead of diverging, geodesics converge and eventually cross. This simple shift introduces three severe algorithmic challenges.
1. Geodesics Wrap Around and Destroy Convexity
On a sphere, if you walk in a straight line (a great circle), you eventually loop back to your starting point. This means that two points on a sphere are connected by multiple geodesics (e.g., traveling clockwise vs. counterclockwise).
As explained by Hyunwoo J. Kim et al. in Manifold-valued Dirichlet Processes, the flat approximation provided by a local tangent space is only valid within a strict boundary called the injectivity radius. Beyond this radius, the mapping between the manifold and the tangent space is no longer unique. This "wrapping" effect fundamentally interferes with global convexity. As Xi Wang et al. note in No-regret Online Learning over Riemannian Manifolds, when geodesics wrap around, they disrupt the very definition of convex objectives, making global online learning guarantees exceptionally difficult to prove.
2. Persistent Metric Distortion Blocks Acceleration
Accelerated gradient methods work by combining the current gradient with a momentum vector from the previous step. In a curved space, this requires parallel transporting a vector from one point on the manifold to another.
In Understanding Riemannian Acceleration via a Proximal Extragradient Framework, Jikai Jin et al. show that positive curvature introduces a persistent metric distortion. Unlike non-positively curved spaces where this distortion vanishes as the algorithm closes in on the optimum, the distortion on a sphere remains stubborn even near the minimizer, acting as a barrier to standard acceleration techniques.
Interestingly, this barrier is distinct from the limits of negative curvature. While Christopher Criscitiello et al. proved a "no-go" theorem in Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles—showing that extreme negative curvature completely blocks acceleration—the converging nature of positive curvature presents a separate, unresolved challenge for acceleration.
3. Projections and Oracles Become Ill-Behaved
Many constrained optimization and online learning algorithms rely on projecting points back onto a feasible set, or using linear optimization oracles to find the best direction forward. On Hadamard manifolds, metric projections are well-behaved contractions (moving two points closer together).
In contrast, Christophe Roux et al. demonstrate in Implicit Riemannian Optimism with Applications to Min-Max Problems that metric projections onto constraint sets become highly unstable and poorly behaved in the presence of positive curvature. This same geometric distortion invalidates the separation and linear optimization oracles used by Zihao Hu et al. in Riemannian Projection-free Online Learning, preventing the extension of projection-free online learning and adaptive tracking frameworks (Minimizing Dynamic Regret on Geodesic Metric Spaces) to spherical geometries.
Non-Positive Curvature (Hadamard) Positive Curvature (Spherical)
(Geodesics Diverge) (Geodesics Converge)
o-------_ _-------o
/ \-_ _-/ \
/ \_ _/ \
o----------------o o----------------o
/ \ / \
o o o o
\ /
\_ _/
\-_ _-/
--------
- Unique shortest geodesics - Multiple intersecting geodesics
- Global convexity holds - Convexity breaks past injectivity radius
- Stable metric projections - Unstable, ill-behaved projections
The Cost of Ignoring Curvature
This theoretical divide has major practical consequences because real-world data and physical systems do not always conform to non-positive curvature constraints.
In robotics, for example, configuration spaces frequently involve continuous rotational joints. Thomas Cohn et al. demonstrated in Non-Euclidean motion planning with graphs of geodesically convex sets that while graph-based motion planning works flawlessly across non-positively curved spaces, the introduction of positive curvature—such as the geometry inherent to 3-DOF ball joints—violates the core convexity assumptions required to guarantee collision-free, globally optimal trajectories.
Similarly, in statistical learning, forcing data into a model with the wrong curvature profile causes severe representation errors. As Salem Said et al. point out in Gaussian Distributions on Riemannian Symmetric Spaces: Statistical Learning With Structured Covariance Matrices, statistical frameworks optimized for non-positive curvature fail when applied to hyperspherical data. If a dataset inherently exhibits positive curvature, models restricted to non-positive spaces cannot accurately capture its variance.
To bypass these limitations, researchers are developing highly specialized architectures. For instance, Xuan Son Nguyen et al. introduced Siegel Neural Networks, designing custom neural network layers tailored to Siegel symmetric spaces. By moving beyond rigid non-positive curvature assumptions, such frameworks allow deep learning models to accurately classify complex, structured data like radar clutter without succumbing to geometric distortion.
Key Foundational Work
Gaussian Distributions on Riemannian Symmetric Spaces: Statistical Learning With Structured Covariance Matrices (2017) — contributed 1 concept elements to this direction.
Manifold-valued Dirichlet Processes (2015) — contributed 1 concept elements to this direction.