Riemannian Optimization for Geometric Constraints

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2243 papers267 conceptsfrozen 2026-10-04

In standard machine learning, we often treat parameters as points in a flat, infinite Euclidean space. We calculate a gradient, take a step, and repeat. However, many real-world problems are defined by parameters that must satisfy strict structural constraints: a camera's rotation must be an orthogonal matrix, a covariance matrix must be symmetric positive definite (SPD), and a neural network's weights might need to stay on a unit sphere.

When we apply standard optimization to these problems, we encounter a "Flat Earth" problem. A linear step in a flat direction immediately "falls off" the valid parameter space. Riemannian optimization solves this by redefining the geometry of the search space itself, ensuring that every step of the algorithm respects the intrinsic constraints of the manifold.

The Geometry of Constraints

Traditional optimization handles constraints using projections—taking a step in flat space and then "squashing" the result back onto the feasible set. As noted by Khurrum Aftab and Richard Hartley in Generalized Weiszfeld Algorithms for Lq Optimization, this approach often ignores the intrinsic geometry of the space, leading to inefficiencies or convergence failures in tasks like 3D reconstruction.

Instead, we treat the feasible set as a Riemannian manifold: a curved surface that looks locally like Euclidean space. By viewing constraints as the "shape" of the world, we can develop algorithms that never leave the manifold in the first place. This is particularly critical for structures like the Stiefel manifold (orthogonal matrices) or the Grassmann manifold (subspaces), which appear frequently in low-rank matrix completion and subspace learning, as discussed by Jiang Hu and Na Li in Decentralized Riemannian natural gradient methods with Kronecker-product approximations.

Moving on a Curve: Retractions and Geodesics

In Euclidean space, we update a position $x$ by adding a gradient $v$: $x_{new} = x + v$. On a manifold, this addition is undefined. To move while staying on the surface, we use two primary tools:

  1. The Tangent Space: At any point on the manifold, there is a "flat" plane of all possible directions we could move. We compute our gradients here.
  2. Retractions: A retraction is a mapping that takes a vector from the tangent space and "wraps" it back onto the manifold.

While the mathematically "perfect" way to move is along a geodesic (the shortest path on a curved surface), geodesics are often computationally expensive to calculate. Chunming Tang and Jinbao Jian demonstrate in A descent method for nonsmooth multiobjective optimization problems on Riemannian manifolds that using retractions provides a much more efficient way to find optimal solutions without losing the benefits of the manifold's structure.

The Challenge of Memory: Vector Transport

One of the most powerful tools in modern optimization is momentum—using past gradients to speed up current progress. In flat space, this is easy: you just add the old gradient to the new one.

On a manifold, this is impossible. A gradient calculated at point $A$ lives in the tangent space of $A$. If we move to point $B$, the old gradient is now "floating in the air," disconnected from the tangent space of $B$. To fix this, we use Vector Transport.

As zhiqiang xu and Yiping Ke explain in Stochastic Variance Reduced Riemannian Eigensolver, vector transport allows us to "slide" a gradient along the manifold from one point to another. This mechanism is what enables advanced techniques like Riemannian SVRG (Hongyi Zhang and Suvrit Sra, Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds) to achieve exponential convergence rates by properly managing variance reduction across different tangent spaces.

Beyond Closed Forms: Data-Driven Manifolds

Most classical work assumes the manifold is defined by a known equation (like $x^T x = 1$). However, recent research has pushed into implicit manifolds—surfaces defined only by data samples.

In Landing with the Score: Riemannian Optimization through Denoising, Andrey Kharitenko and Niao He show how to use the "score function" from diffusion models to recover geometric operations. This allows us to optimize objective functions while strictly adhering to complex, data-driven constraints that don't have a simple mathematical formula. Similarly, Natasa Tagasovska and Andreas Loukas use this concept in PropEn to enhance properties of scientific designs while ensuring they stay within the realistic "data manifold."

Why It Works: Stability and Efficiency

By embedding the constraints into the geometry of the space, Riemannian algorithms gain several advantages:

Ultimately, Riemannian optimization turns a "constrained problem" into an "unconstrained problem on a curved space." By respecting the natural shape of the data, these algorithms provide a more robust and principled framework for modern machine learning.

Papers in this version

Showing 60 of 243 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 293 papers
  2. Version 1 · 2026-10-04 · 243 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_manifold_constrained_optimization_optimi_27903bf5,
  title = {manifold-constrained optimization (optimization on curved spaces)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manifold-constrained-optimization-optimization-on-curved-spaces/rdr_27903bf5ca9d4028be8d120a0307912e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_riemannian_optimization_for_geometric_co_27903bf5_v1,
  title = {Riemannian Optimization for Geometric Constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/riemannian-optimization-for-geometric-constraints/rdr_27903bf5ca9d4028be8d120a0307912e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}