In standard machine learning, we typically assume that parameters live in a flat, unconstrained Euclidean space ($\mathbb{R}^n$). When optimizing a loss function, we compute a gradient and take a straight-line step in that direction.
However, many real-world problems impose strict structural constraints on parameters. For instance, variables might need to remain orthogonal (such as in orthogonal neural networks or Principal Component Analysis), stay positive-definite (like covariance matrices), or preserve hierarchical relationships (as in hyperbolic embeddings). These constrained search spaces form smooth, curved surfaces known mathematically as Riemannian manifolds.
Directly applying standard Euclidean optimization to these spaces fails. If you take a linear step along a gradient vector from a point on a sphere, you immediately fly off the surface, violating the constraints. While one could project the parameters back onto the valid space after every step, this approach ignores the intrinsic geometry of the space, often leading to poor convergence or numerical instability.
Riemannian optimization solves this by redefining optimization entirely from the perspective of the curved surface itself.
The Core Mechanics: Tangent Spaces and Retractions
To understand how optimization works on a curved surface, we must replace flat-space operations with geometric equivalents. The framework relies on three fundamental concepts: the tangent space, the Riemannian gradient, and retractions.
Tangent Space T_x M (Flat plane touching at x)
^
| \ Riemannian Gradient (Direction of step)
| \
| v
--------------+-------*------------------
/ \
/ \ Retraction (Snaps vector back to surface)
/ v
x ------------> x_next
Riemannian Manifold M (Curved Surface)
1. The Tangent Space ($T_xM$)
Imagine standing on the surface of the Earth. Locally, the ground feels flat. This local linearization is the tangent space ($T_xM$) at your current position $x$. The tangent space is a true Euclidean vector space, meaning we can perform standard vector arithmetic inside it. Every point on a manifold has its own unique tangent space.
2. The Riemannian Gradient
A standard Euclidean gradient points in the direction of steepest ascent in ambient space, which often points away from the manifold. To find the direction of steepest ascent while staying on the surface, we project the Euclidean gradient orthogonally onto the local tangent space. This projection yields the Riemannian gradient. As noted by Nicolas Boumal et al. in their work on the Manopt toolbox, translating these derivatives systematically is essential for making manifold optimization practical.
3. Retractions
Once we have a search direction in the tangent space, we take a step inside this flat plane. However, this step moves us away from the curved manifold. A retraction ($R_x$) is a smooth mapping that "snaps" the vector from the tangent space back down onto the manifold.
While the mathematically ideal way to move along a manifold is via a geodesic (the shortest path along the curved surface), computing geodesics exactly can be computationally prohibitive. Retractions serve as a first-order approximation of geodesics, ensuring computational efficiency while strictly satisfying the manifold's constraints.
Moving Across Spaces: Vector Transport
First-order optimization methods like Gradient Descent only require a local gradient and a retraction. However, modern optimization relies heavily on historical information—such as momentum in SGD or moving averages of squared gradients in Adam.
This creates a fundamental geometric conflict. If you move from point $x_t$ to $x_{t+1}$, your historical momentum vector resides in the tangent space $T_{x_t}M$. But your new gradient resides in $T_{x_{t+1}}M$. Because these are two entirely different vector spaces, you cannot add them together.
To resolve this, Riemannian optimization introduces vector transport. Vector transport takes a vector from one tangent space and moves it to another along a specified path, adjusting its orientation to respect the curvature of the manifold.
This mechanism allows researchers to port advanced Euclidean optimizers to curved spaces:
- Quasi-Newton Methods: Wen Huang et al. utilized vector transport operators in A Broyden Class of Quasi-Newton Methods for Riemannian Optimization to properly update and transport Hessian approximations across curved spaces.
- Conjugate Gradient Methods: Hiroyuki Sakai et al. demonstrated in Global Convergence of Hager-Zhang type Riemannian Conjugate Gradient Method how proper vector transport guarantees global convergence when computing conjugate search directions on spheres.
- Adaptive Algorithms: To implement adaptive learning rates on matrix manifolds, Hiroyuki Kasai et al. developed the RASA framework, which tracks row and column variances of gradients by transporting coordinate-wise history across iterations.
Why Geodesic Convexity Matters
In flat spaces, we rely on standard convexity to guarantee that local minima are global minima. On curved spaces, standard convexity breaks down because the line segment connecting two points typically leaves the manifold.
Instead, the field relies on geodesic convexity. A function is geodesically convex if, for any two points on the manifold, the function remains convex along the geodesic path connecting them. As explored by Hongyi Zhang et al. in First-order Methods for Geodesically Convex Optimization, establishing this property allows first-order methods to achieve provable convergence rates analogous to their Euclidean counterparts, even when the underlying space exhibits non-zero curvature.
Solving Advanced Optimization Challenges
As machine learning scales, the challenges of non-convex optimization—such as saddle points, decentralized communication, and meta-learning—must be re-engineered for Riemannian spaces.
Escaping Saddle Points
In non-convex optimization, saddle points can stall gradient descent. In Euclidean spaces, adding isotropic random noise helps the optimizer escape along a descent direction. On manifolds, injecting noise requires care; the perturbation must be generated directly within the local tangent space and then mapped to the manifold. Christopher Criscitiello et al. formalized this in Efficiently escaping saddle points on manifolds, proving that manifold-constrained noise injection can efficiently find escape routes without leaving the parameter space.
Min-Max and Decentralized Optimization
More complex optimization landscapes, such as those found in game-theoretic formulations or multi-agent systems, introduce further geometric complications:
- Min-Max Optimization: For adversarial or zero-sum formulations on manifolds, Andi Han et al. extended the theory to non-convex non-concave regimes in Nonconvex-nonconcave min-max optimization on Riemannian manifolds, introducing second-order algorithms to stabilize local minimax points.
- Decentralized Consensus: In distributed networks where agents must agree on a parameter state while constrained to a manifold (e.g., decentralized PCA), standard linear averaging violates the constraints. Shixiang Chen et al. addressed this in Decentralized Riemannian Gradient Descent on the Stiefel Manifold by combining Riemannian gradient steps with specialized gradient tracking to maintain consensus across agents without breaking orthogonality constraints.
Meta-Optimization via Implicit Differentiation
When training neural networks to act as optimizers on manifolds, standard meta-learning requires "unrolling" the optimization path, which demands massive memory. To bypass this, Xiaomeng Fan et al. introduced Efficient Riemannian Meta-Optimization by Implicit Differentiation. By leveraging the implicit function theorem directly on the manifold's tangent spaces, the authors decoupled meta-gradient computations from the optimization trajectory length, drastically reducing memory overhead.
Conclusion
Riemannian optimization shifts the burden of constraint enforcement from an ad-hoc post-processing step (like projection) to the foundational geometry of the optimizer itself. By leveraging tangent spaces for vector arithmetic, retractions for stepping onto surfaces, and vector transport for tracking history, it provides a mathematically rigorous framework for optimizing over non-Euclidean parameter spaces.