Infinity-Wasserstein Distributionally Robust Principal Component Analysis: Exact Reformulation, Finite-Sample Guarantee, and Efficient Algorithm

Zhouxing Luo, Lei Wang, Xin LiuarXiv 2026 · arXiv

TL;DR

This paper introduces InfWPCA, a new framework for Principal Component Analysis that is robust against data noise and outliers by using the infinity-Wasserstein distance. Unlike previous robust models, InfWPCA provides an exact mathematical reformulation that is computationally efficient, offers rigorous statistical guarantees for heavy-tailed data, and significantly outperforms standard PCA in real-world tasks like image classification and portfolio management.

What Problem Does This Paper Address?

Principal Component Analysis (PCA) is the "Swiss Army knife" of dimension reduction. However, it has a well-known Achilles' heel: it is incredibly sensitive to outliers and sampling errors. If your training data is slightly "off" from the true distribution, the subspace PCA finds might be useless for future data.

Researchers have tried to fix this using Distributionally Robust Optimization (DRO). The idea is to optimize for the "worst-case" distribution within a certain "distance" (usually the Wasserstein distance) of your observed data. But there’s a catch-22 in existing work:

  1. The "Boring" Result: Using the standard order-2 Wasserstein distance ($W_2$) often results in the exact same subspace as regular PCA—it just changes the value of the objective function without actually making the model more robust.
  2. The "Impossible" Result: More complex models that do change the subspace often lack an "exact reformulation," meaning they are computationally nightmarish to solve and lack theoretical guarantees.

The authors ask: Can we build a DRO-PCA model that is both nontrivial (finds a better, more robust subspace) and tractable (can be solved efficiently and exactly)?

Methods

The authors turn to the infinity-Wasserstein distance ($W_\infty$). While $W_2$ constrains the average cost of moving data points, $W_\infty$ puts a hard cap on the maximum displacement of any single point.

Key Technical Innovations:

  • Exact Reformulation: They prove that the infinite-dimensional "min-max" problem (minimizing error over the worst-case distribution) can be collapsed into a finite-dimensional minimization problem. The new objective function looks at individual sample residuals: $\frac{1}{n} \sum (\|residual_i\| + \rho)^2$.
  • Beyond Covariance: Standard PCA only cares about the global covariance matrix. The InfWPCA objective retains "samplewise residual information," allowing the model to be sensitive to specific outliers that a covariance matrix would normally smooth over.
  • IRWPCA Algorithm: They developed the Iteratively Reweighted Wasserstein PCA algorithm. It uses a "majorization-minimization" approach, which essentially solves a sequence of weighted PCA problems. It’s simple to implement and comes with a proof of convergence.
  • Worst-Case Distribution: The paper explicitly constructs the "worst-case" distribution that the model is protecting against—a rare feat in DRO research that provides deep structural insight.

Datasets

The authors evaluated their model and algorithm on:

  1. MNIST: The classic handwritten digit dataset (3,000 images), used to test representation and classification under three types of noise: Gross Gaussian noise, Replacement noise, and Sparse impulse noise.
  2. Fama-French 48 Industry Portfolios: Real-world financial data consisting of 674 months of returns across 48 industries, used to test the model's effectiveness in minimum-variance portfolio selection.

Key Insights and Ideas

  • The $W_\infty$ "Sweet Spot": The infinity-Wasserstein distance is the "Goldilocks" metric here. It is strong enough to force the PCA to pick a different, more stable subspace, but structured enough to allow for an exact mathematical solution.
  • Robustness for Heavy Tails: Most statistical proofs require "sub-Gaussian" data (thin tails). This paper only requires a finite fourth moment. This means the model is theoretically guaranteed to work even on "messy" real-world data with heavy tails (like financial returns).
  • Samplewise Weighting: The algorithm naturally places more weight on samples with smaller residuals. This "reweighting" mechanism is what allows the model to ignore or mitigate the influence of distant outliers.

Key Results

  • Computational Superiority: The IRWPCA algorithm was significantly faster and reached lower objective values than general-purpose Riemannian optimization methods (like RSG or DSGM), especially as the target dimension ($r$) increased.
  • Noise Resilience: On MNIST with 50% noise, InfWPCA maintained much higher classification accuracy and lower reconstruction error (MSE) than classical PCA and other Wasserstein-based competitors.
  • Financial Gains: In the portfolio selection task, InfWPCA achieved a Sharpe ratio of 0.835, outperforming classical PCA (0.816) and other robust variants. It also showed the lowest "maximum drawdown," meaning it managed risk better during market downturns.

Limitations and Future Work

  • Structural Constraints: The current model focuses on the standard Stiefel manifold. The authors note that adding extra constraints, like sparsity (Sparse PCA) or non-negativity, would be a valuable next step.
  • Task-Awareness: Currently, the PCA is performed as a standalone step. The authors suggest a "task-aware" approach where the robust subspace selection is guided directly by the needs of the downstream application (e.g., specific regression or classification goals).
  • Parameter Sensitivity: While the model is robust, the "uncertainty radius" ($\rho$) still needs to be tuned (usually via cross-validation), which adds a layer of complexity to the pipeline.

Summary updated

Abstract

Principal component analysis (PCA) is a fundamental and ubiquitous technique for dimension reduction and representation learning, yet its empirical formulation remains vulnerable to distributional uncertainty. While distributionally robust optimization (DRO) offers a natural remedy, existing order-2 Wasserstein models of PCA either preserve the nominal PCA subspace or lack an exact reformulation. This paper demonstrates how the infinity-Wasserstein distance can reconcile these two desirable properties. For the resulting DRO model of PCA, we derive an exact finite-dimensional reformulation and explicitly identify a worst-case distribution attaining the inner maximum. Crucially, the reformulated objective retains samplewise residual information beyond the empirical covariance matrix, revealing a mechanism through which distributional robustness can reshape the learned principal subspace. Under the mild assumption that the true distribution has a finite fourth moment, we further establish a uniform finite-sample performance guarantee for the proposed model. An iteratively reweighted majorization-minimization algorithm is also developed to solve the resulting nonconvex and nonsmooth optimization problem, with a rigorous analysis of its convergence properties. Numerical experiments substantiate the computational efficiency of the proposed algorithm and the practical effectiveness of the infinity-Wasserstein model in downstream applications.
BibTeX
@misc{luo2026infinity,
  author = {Zhouxing Luo and Lei Wang and Xin Liu},
  title = {{Infinity-Wasserstein Distributionally Robust Principal Component Analysis: Exact Reformulation, Finite-Sample Guarantee, and Efficient Algorithm}},
  year = {2026},
  eprint = {2611.23877},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23877},
}
Figure