Manifold to Manifold Dimensionality Reduction

51 concepts45 papers1 proposal

When dealing with complex data like video trajectories, brain-computer interface (BCI) signals, or deep feature correlations, we often represent data points not as flat vectors, but as structured matrices. These matrices frequently live on non-Euclidean, curved spaces known as Riemannian manifolds. Examples include Symmetric Positive Definite (SPD) matrices (such as covariance matrices), Grassmann manifolds (spaces of linear subspaces), and Lie groups (spaces representing continuous transformations).

Standard dimensionality reduction techniques like PCA, or traditional neural network layers, assume data lives in a flat Euclidean space. If you flatten a covariance matrix into a vector and apply a standard linear projection, you destroy its geometric properties. For instance, a compressed covariance matrix might lose its positive definiteness, rendering its eigenvalues negative and stripping it of its physical meaning.

To solve this, a powerful paradigm has emerged: manifold-to-manifold dimensionality reduction. Instead of flattening the data, these methods compress high-dimensional manifolds directly into lower-dimensional manifolds of the exact same mathematical type, preserving their intrinsic geometry and curvature.

The Cost of Flattening: The Swelling Effect and Structural Distortion

Why can't we just treat matrix manifolds as flat spaces? The core issue lies in how distances and geometry behave on a curved surface.

Consider high-dimensional SPD matrices used in neuroimaging or image classification. As Bouchard et al. observe, conventional dimensionality reduction fails to maintain the intrinsic geometric relationships of the underlying manifold, leading to numerical instability when processing high-density sensor data.

A classic symptom of this mismatch is the swelling effect. When you perform operations like averaging or linear projection on SPD matrices using Euclidean geometry, the determinant (which represents the volume of the data's dispersion) can become artificially inflated. As Chevallier et al. demonstrate, projecting SPD data onto a curved manifold rather than a flat Euclidean plane prevents this swelling effect, ensuring that low-dimensional visualizations and classifications remain mathematically consistent with the original high-dimensional data.

Euclidean Projection:  SPD Matrix (High-Dim) ───[ Flatten & Project ]───> Defective Matrix (May lose positive-definiteness)
Manifold Projection:   SPD Matrix (High-Dim) ───[ Bilinear Mapping ]────> SPD Matrix (Low-Dim, Preserves Geometry)

The Core Mechanism: Bilinear Mappings on Stiefel Manifolds

To reduce the dimensions of an $n \times n$ matrix $X$ down to a smaller $d \times d$ matrix $Y$ (where $d < n$) without leaving the manifold, researchers utilize a bilinear transformation:

$$Y = W^T X W$$

Here, $W$ is an $n \times d$ transformation matrix. Crucially, to ensure that the geometric properties of the original space are inherited by the lower-dimensional representation, $W$ cannot be just any matrix; it is constrained to be semi-orthogonal, meaning $W^T W = I_d$. The space of these semi-orthogonal matrices is itself a well-known manifold called the Stiefel manifold.

Why does this specific mechanism work? Let's look at the intuition for SPD matrices:

  1. Preserving Positive Definiteness: An SPD matrix $X$ satisfies $v^T X v > 0$ for any non-zero vector $v$. If we transform it to $Y = W^T X W$, then for any non-zero vector $u$, we have $u^T Y u = u^T W^T X W u = (Wu)^T X (Wu)$. As long as $W$ has full column rank, $Wu$ is non-zero, meaning $(Wu)^T X (Wu) > 0$. Thus, $Y$ is guaranteed to remain on the SPD manifold.
  2. Preserving Scalar Products: As early work by Harmeling et al. established, maintaining scalar product preservation during dimensionality reduction ensures that the geometric properties of the feature space are inherited by the lower-dimensional representation, allowing Euclidean algorithms to be kernelized directly in the projected space.

This bilinear strategy forms the backbone of modern geometric deep learning. In SPDNet, introduced by Van Gool et al., specialized bilinear mapping layers reduce the dimensionality of SPD matrices by learning weight matrices constrained to a Stiefel manifold. To introduce non-linearity without destroying the manifold structure, Jia et al. and Ju et al. utilize eigenvalue rectification layers (similar to a ReLU but applied exclusively to the matrix's eigenvalues), ensuring the output remains strictly positive definite throughout deep architectures.

Extending to Subspaces: Grassmannian Dimensionality Reduction

The same principle applies when data points are represented as linear subspaces rather than covariance matrices. A space of $p$-dimensional linear subspaces in $\mathbb{R}^n$ forms a Grassmann manifold.

To compress a high-dimensional subspace to a lower-dimensional one, we learn a discriminative mapping directly between two Grassmann manifolds. For instance, Wang et al. extended the classic Locality Preserving Projections (LPP) framework to non-Euclidean spaces by applying a linear transformation followed by a QR decomposition. This step ensures that the resulting low-dimensional representatives satisfy the strict orthogonality requirements of the target Grassmann manifold.

By avoiding an intermediate embedding into a high-dimensional Hilbert space, this geometry-aware reduction preserves the discriminative structural information of the data while drastically reducing computational overhead in video and face verification tasks, as shown by Luo et al..

Broadening the Geometric Horizon

The success of geometry-preserving dimensionality reduction has led to its adoption across a wide variety of mathematical spaces:

  • Lie Groups: For skeleton-based action recognition, human joint rotations can be modeled as trajectories on Lie groups. Huang et al. designed pooling operations that select elements based on their rotation angle magnitude, reducing spatial and temporal dimensionality while keeping the output within the valid Lie group.
  • Siegel Spaces: For complex radar clutter and node classification, Nguyen et al. designed Siegel Neural Networks, which map high-dimensional points through the Stiefel manifold to a lower-dimensional Siegel space, preserving its intricate quotient structure.
  • Preserving Extreme Points: Beyond deep learning, orthogonal transformations like QR decomposition are preferred over random projections in matrix factorization. Benson demonstrates that these transformations maintain the relative positions of extreme points, ensuring that the boundaries of a dataset's conical hull remain perfectly identifiable even after massive data compression.

By treating the target space not as a flat canvas but as a structured, curved manifold, these techniques allow machine learning models to compress complex data while respecting the physical and mathematical laws governing the original features.

Key Foundational Work

Kernel Feature Spaces and Nonlinear Blind Souce Separation (2001) — contributed 1 concept elements to this direction.
On d-dimensional d-semimetrics and simplex-type inequalities for high-dimensional sine functions (2009) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_manifold_preserving_dimensionality_reduc_52e8c76d,
  title = {manifold-preserving dimensionality reduction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manifold-preserving-dimensionality-reduction/rdr_52e8c76d0a064810ba6420c341689c42},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_manifold_preserving_dimensionality_reduc_52e8c76d_v1,
  title = {manifold-preserving dimensionality reduction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manifold-preserving-dimensionality-reduction/rdr_52e8c76d0a064810ba6420c341689c42/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake