The Manifold Hypothesis is a cornerstone of modern machine learning: it posits that high-dimensional real-world data (such as images, sensor arrays, or text embeddings) actually concentrates around a much lower-dimensional, curved geometric surface.
However, real-world data collection is rarely pristine. It is frequently corrupted by off-manifold noise and spiky outliers. As noted by Dian Gong et al. in Robust Multiple Manifold Structure Learning, these irregularities can pull an estimated manifold completely out of its true shape. When standard manifold learning algorithms encounter these anomalies, their geometric representations distort. For instance, in algorithms like Isomap, a single noisy data point sitting between two distinct folds of a manifold can create a "short-circuit," destroying the low-dimensional mapping (Tianqiang Huang et al. in Nonlinear dimensionality reduction algorithm and application to hospital performance evaluation).
To build algorithms that can uncover the true underlying geometry of dirty data, researchers have increasingly turned to manifold-aware robust estimators, specifically replacing classical averages with geometric medians.
Why Squared Distances Fail in Curved Spaces
To understand why manifolds distort under noise, we must look at how central tendencies are calculated. In flat Euclidean space, the standard sample mean minimizes the sum of squared Euclidean distances:
$$\mu = \arg\min_{\mu} \sum_{i=1}^n \|x_i - \mu\|^2$$
When moving to non-Euclidean spaces—such as the Riemannian manifold of Symmetric Positive Definite (SPD) matrices or the Grassmannian manifold of subspaces—the natural extension of the mean is the Fréchet Mean, which minimizes the sum of squared geodesic (intrinsic, curved) distances.
However, squaring the distance gives outliers disproportionate leverage. If an anomaly or an off-manifold noise point appears far away from the true data distribution, its squared distance grows quadratically. To minimize the overall loss, the Fréchet Mean is dragged drastically toward the outlier. As JooSeuk Kim et al. point out in On the Robustness of Kernel Density M-Estimators, standard averaging techniques degrade rapidly when observations are noisy or nonhomogeneous.
The Geometric Median as a Manifold-Valued L1-Norm
To achieve robustness, a powerful alternative is the Fréchet Median (or Geometric Median). Instead of minimizing squared distances, the geometric median minimizes the sum of unsquared geodesic distances:
$$m = \arg\min_{m} \sum_{i=1}^n d(x_i, m)$$
Mathematically, this shift from squared to unsquared distances acts as a manifold-valued $L_1$-norm. In A Nonlinear Regression Technique for Manifold Valued Data with Applications to Medical Image Analysis, Monami Banerjee et al. demonstrate that this mechanism provides exceptional stability against high noise levels.
The intuition behind this robustness lies in the gradient of the loss function. For squared distances, the gradient (the "pull" an item exerts on the estimator) grows linearly with distance. An outlier infinitely far away exerts infinite pull. For unsquared distances, the gradient depends only on the direction of the data point relative to the median, not its distance:
$$\nabla d(x_i, m) = -\frac{\exp_m^{-1}(x_i)}{d(x_i, m)}$$
Because the denominator normalizes the distance, an outlier's influence is strictly bounded. No matter how far an anomaly is pushed off the manifold, its pull on the geometric median remains constant, allowing the estimator to stay anchored within the dense, clean region of the true manifold.
Robustness in Action: Covariances and Subspaces
This geometric framework has proven highly effective across specialized mathematical structures, notably in signal processing and computer vision.
1. Matrix Manifolds for Signal Processing
In radar and communication systems, background environmental noise (clutter) is highly unpredictable. Xiaoqiang Hua demonstrates in MIG median detectors with manifold filter that calculating geometric medians on the manifold of positive-definite matrices prevents interference spikes from skewing the estimated covariance structure.
Because standard statistical averages degrade linearly as interference increases, shifting to the geometric median allows signal detectors to maintain stable background estimation. Similarly, Yusuke Ono shows in Towards a median signal detector through the total Bregman divergence and its robustness analysis that geometric medians on Hermitian Positive-Definite (HPD) manifolds preserve the underlying physical correlation of complex signals even when surrounded by high-power, non-stationary noise.
2. Grassmann Manifolds for Subspace Learning
When data points represent entire linear subspaces (points on a Grassmannian manifold), finding a representative prototype is highly vulnerable to corrupted frames or pixel blocks. Rudrasis Chakraborty et al. show in Intrinsic Grassmann Averages for Online Linear and Robust Subspace Learning that replacing the manifold mean with a Fréchet Median yields inherent outlier resistance without the heavy computational overhead of traditional $L_1$ optimization frameworks.
To make this practical for real-time applications, Nathan Mankovich et al. introduce the flag median and the FlagIRLS algorithm in The Flag Median and FlagIRLS, providing an iteratively reweighted least squares approach that converges rapidly to a stable prototype subspace even when a large portion of the dataset consists of irrelevant or noisy samples.