A Bias Trick for Centered Robust Principal Component Analysis (Student Abstract)

Baokun He, Guihong Wan, Haim SchweitzerAAAI 2020 · DOI

Introduction

Principal Component Analysis (PCA) is a fundamental tool in data science for identifying dominant patterns in high-dimensional datasets. However, standard PCA is highly sensitive to outliers; even a small number of anomalous data points can significantly distort the resulting principal components. Robust PCA (RPCA) was developed to mitigate this by identifying and isolating outliers during the dimensionality reduction process.

The Centering Dilemma

A critical requirement for PCA is that the data must be centered around its mean. In a robust context, this means centering the data based only on the "clean" points (non-outliers). This creates a circular dependency: to identify outliers accurately, the data should be centered, but to center the data correctly, one must first identify the outliers. Existing methods often bypass this by centering the entire dataset (including outliers) or using heuristics that lack optimality guarantees.

The Bias Trick

To resolve this "chicken-and-egg" problem, the authors propose a mathematical workaround termed the Bias Trick. Instead of iteratively estimating the mean, they augment the input data matrix $X$ by adding a new coordinate with a large constant value $b$ (the "bias") to every data point.

By running a standard uncentered RPCA algorithm on this augmented space, the algorithm naturally identifies the "bias" as the primary direction of variance. The subsequent $k$ eigenvectors then correspond to the centered principal components of the original data. This approach allows any uncentered RPCA algorithm to be converted into an optimal centered version.

The relationship between the accuracy parameter $\epsilon$ and the eigenvalue error $E_{eigenvalues}$ demonstrates the trade-off involved in selecting the bias value $b$.

Experimental Evaluation

The authors evaluated their algorithm, COPT, against established competitors like Outlier-Pursuit and CoP using several datasets from the UCI Machine Learning Repository, including the Iris, Smoking, WDBC, and Wine datasets.

Key Results

  • Reduced Reconstruction Error: COPT consistently achieved lower average reconstruction errors ($E_{rpca}$) for non-outlier points. On the wdbc dataset, for instance, COPT reduced the error to ~241, compared to ~304 for Outlier-Pursuit.
  • Superior Outlier Detection: Qualitative analysis of the Wine dataset revealed that COPT more accurately identifies outliers at the extreme margins of the data distribution. While competing methods sometimes flagged points near the center or missed those on the fringes, COPT's centering-aware approach proved more reliable.
Visualization of the Wine dataset projected onto principal components $v_1$ and $v_3$, with outliers (red) correctly identified at the margins of the distribution.

Conclusion and Future Work

The Bias Trick provides a simple yet powerful wrapper for robust data analysis, offering optimality with respect to centering for the first time. While the method introduces a slight computational overhead and requires choosing an appropriate bias value $b$, the gains in robustness and accuracy are significant. Future research may extend this trick to other dimensionality reduction techniques that currently struggle with uncentered, noisy data.

Summary updated

BibTeX
@inproceedings{he2020bias,
  author = {Baokun He and Guihong Wan and Haim Schweitzer},
  title = {{A Bias Trick for Centered Robust Principal Component Analysis (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7175},
}
Figure