Rethinking PCA Through Duality
Lists
Only you see this
Abstract
Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-convex (DC) framework, we present several novel formulations and provide new theoretical insights. In particular, we show the kernelizability and out-of-sample applicability for a PCA-like family of problems. Moreover, we uncover that simultaneous iteration, which is connected to the classical QR algorithm, is an instance of the difference-of-convex algorithm (DCA), offering an optimization perspective on this longstanding method. Further, we describe new algorithms for PCA and empirically compare them with state-of-the-art methods. Lastly, we introduce a kernelizable dual formulation for a robust variant of PCA that minimizes the $l_1$ deviation of the reconstruction errors.
BibTeX
@inproceedings{quan2025rethinking,
author = {Jan Quan and Johan Suykens and Panagiotis Patrinos},
title = {{Rethinking PCA Through Duality}},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025},
url = {https://openreview.net/forum?id=IFQBrEAuQ6},
}| Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast Algorithms | arXiv · 2023 |
| L1-norm Kernel PCA | arXiv · 2017 |
| Non-Greedy L21-Norm Maximization for Principal Component Analysis | arXiv · 2016 |
| Deep Kernel Principal Component Analysis for multi-level feature learning | Neural Networks · 2024 |
| Dual Principal Component Pursuit | arXiv · 2015 |
| Robust PCA in High-dimension: A Deterministic Approach | arXiv · 2012 |