$e^{\text{RPCA}}$: Robust Principal Component Analysis for Exponential Family Distributions

Xiaojun Zheng, Simon Mak, Liyan Xie, Yao XiearXiv 2023 · DOI · arXiv

Abstract

Robust Principal Component Analysis (RPCA) is a widely used method for recovering low-rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low-rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non-Gaussian. We thus propose a new method called Robust Principal Component Analysis for Exponential Family distributions ($e^{\text{RPCA}}$), which can perform the desired decomposition into low-rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient $e^{\text{RPCA}}$ decomposition. The effectiveness of $e^{\text{RPCA}}$ is then demonstrated in two applications: the first for steel sheet defect detection, and the second for crime activity monitoring in the Atlanta metropolitan area.

Introduction

The goal of Robust Principal Component Analysis (RPCA) is to decompose a data matrix $M$ into a low-rank component $L$ (representing stable background structure) and a sparse component $S$ (representing anomalies or outliers). While standard RPCA is a powerful tool, it typically assumes Gaussian noise or clean data corrupted only by sparse outliers. In many real-world applications, data follows non-Gaussian distributions, such as binary bits from high-frequency sensors (Bernoulli) or integer counts from crime reports (Poisson). When standard RPCA is applied to such data, it often misinterprets natural distributional variance as sparse anomalies, leading to "cluttered" results and poor background recovery.

Methodology

The authors propose $e^{\text{RPCA}}$, a penalized maximum likelihood estimation framework that integrates exponential family likelihoods directly into the decomposition process. Instead of minimizing a standard distance metric, $e^{\text{RPCA}}$ minimizes the negative log-likelihood of the data given the parameters $\Theta = L + S$.

Key innovations include:

  1. Distribution-Aware Objective: The loss function is tailored to the known distribution of the data (e.g., Poisson loss for count data).
  2. ADMM Optimization: An Alternating Direction Method of Multipliers algorithm efficiently solves the problem by alternating between Singular Value Thresholding for $L$, soft thresholding for $S$, and gradient-based updates for $\Theta$.
  3. Multi-Group Setting: The framework allows for longitudinal analysis where multiple groups share a common low-rank background but have distinct sparse anomalies.

Results

The effectiveness of $e^{\text{RPCA}}$ was demonstrated through synthetic simulations and real-world applications.

Synthetic Simulations

Simulations using Bernoulli, Exponential, and Poisson noise showed that $e^{\text{RPCA}}$ consistently achieves lower recovery errors for both $L$ and $S$ compared to standard RPCA and ePCA. As shown in Figure 4, the performance gap widens as the matrix dimensions increase, highlighting the robustness of the proposed method.

Matrix recovery error across Bernoulli, Exponential, and Poisson distributions.

Steel Sheet Defect Detection

In a manufacturing application using the Severstal dataset, $e^{\text{RPCA}}$ was used to detect physical defects in steel sheets from simulated binary image data. While standard RPCA produced noisy anomaly maps that captured background texture, $e^{\text{RPCA}}$ provided clean maps that pinpointed actual defects like gashes and dents.

Comparison of defect detection on steel sheets using RPCA and $e^{\text{RPCA}}$.

Atlanta Crime Monitoring

Applying $e^{\text{RPCA}}$ to burglary reports in Atlanta allowed for the separation of stable crime patterns from irregular spikes. The multi-group setting revealed that while background patterns are stable, anomalies vary significantly between seasons and between residential and commercial areas.

Conclusion

$e^{\text{RPCA}}$ provides a robust framework for matrix decomposition in the presence of non-Gaussian noise. By incorporating probabilistic constraints from the exponential family, it significantly improves the accuracy of anomaly detection and background recovery. Future work will address computational bottlenecks in the SVT step and explore Bayesian extensions for uncertainty quantification.

Summary updated

Abstract

Robust Principal Component Analysis (RPCA) is a widely used method for recovering low-rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low-rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non-Gaussian. We thus propose a new method called Robust Principal Component Analysis for Exponential Family distributions ($e^{\text{RPCA}}$), which can perform the desired decomposition into low-rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient $e^{\text{RPCA}}$ decomposition. The effectiveness of $e^{\text{RPCA}}$ is then demonstrated in two applications: the first for steel sheet defect detection, and the second for crime activity monitoring in the Atlanta metropolitan area.
BibTeX
@misc{zheng2023e,
  author = {Xiaojun Zheng and Simon Mak and Liyan Xie and Yao Xie},
  title = {{$e^\{\textbackslash\{\}text\{RPCA\}\}$: Robust Principal Component Analysis for Exponential Family Distributions}},
  year = {2023},
  eprint = {2310.19787},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.19787},
}
Figure