$e$PCA: High dimensional exponential family PCA

Lydia T. Liu, Edgar Dobriban, Amit SingerThe Annals of Applied Statistics 2018 · DOI · arXiv

Abstract

Many applications involve large datasets with entries from exponential family distributions. Our main motivating application is photon-limited imaging, where we observe images with Poisson distributed pixels. We focus on X-ray Free Electron Lasers (XFEL), a quickly developing technology whose goal is to reconstruct molecular structure. In XFEL, estimating the principal components of the noiseless distribution is needed for denoising and for structure determination. However, the standard method, Principal Component Analysis (PCA), can be inefficient in non-Gaussian noise. Motivated by this application, we develop $e$PCA (exponential family PCA), a new methodology for PCA on exponential families. $e$PCA is a fast method that can be used very generally for dimension reduction and denoising of large data matrices with exponential family entries. We conduct a substantive XFEL data analysis using $e$PCA. We show that $e$PCA estimates the PCs of the distribution of images more accurately than PCA and alternatives. Importantly, it also leads to better denoising. We also provide theoretical justification for our estimator, including the convergence rate and the Marchenko–Pastur law in high dimensions. An open-source implementation is available.

Introduction

Principal Component Analysis (PCA) is a foundational tool in data science, widely used for visualization, dimensionality reduction, and denoising. However, standard PCA is mathematically optimized for Gaussian noise. In many modern high-dimensional datasets, such as those found in genomics or photon-limited imaging, noise follows different distributions (e.g., Poisson or Binomial). This leads to a "Gaussian bias" where PCA struggles with heteroskedastic noise—noise levels that change depending on the signal strength.

In X-ray Free Electron Laser (XFEL) imaging, for instance, detectors count individual photons, resulting in Poisson-distributed noise. Standard PCA often fails to distinguish between high-variance signal and high-variance noise in these contexts. While iterative likelihood-based methods exist for exponential family PCA, they are often computationally prohibitive for large datasets and lack convergence guarantees.

Noisy photon counts observed in XFEL imaging, illustrating the challenge of Poisson noise.

Methodology

The $e$PCA framework introduces a fast, deterministic four-step pipeline that avoids the pitfalls of iterative optimization:

  1. Diagonal Debiasing: Utilizing the known mean-variance relationship of the exponential family (e.g., variance equals mean in Poisson distributions), the algorithm subtracts the estimated noise variance from the diagonal of the sample covariance matrix.
  2. Homogenization: This critical step rescales the data so that the noise component has unit variance across all features, preventing the signal from being dampened by standard normalization techniques.
  3. Eigenvalue Shrinkage: By applying the Marchenko–Pastur law from Random Matrix Theory, $e$PCA identifies and "shrinks" eigenvalues that are likely attributable to noise.
  4. Heterogenization & Scaling: The matrix is transformed back to its original scale with a final adjustment factor to correct for biases introduced during the pipeline.

Results and Discussion

The authors validated $e$PCA using synthetic simulations and real-world datasets, including XFEL diffraction maps and Human Genome Diversity Project (HGDP) SNP data.

Superior Denoising and Structure Discovery

In XFEL simulations, $e$PCA (combined with an Empirical Best Linear Predictor) produced significantly clearer images and lower Mean Squared Error (MSE) compared to standard PCA. On genetic datasets, $e$PCA revealed clearer population structures, whereas standard PCA was often skewed by outliers. Notably, the authors proved that for genetic data, their homogenization technique is mathematically equivalent to Hardy-Weinberg Equilibrium (HWE) normalization, providing a rigorous theoretical basis for a long-standing empirical practice in genetics.

Comparison of denoising performance: (a) Clean intensity maps, (b) Noisy photon counts, (c) Standard PCA denoising, and (d) ePCA + EBLP denoising.

Computational Efficiency

A major advantage of $e$PCA is its speed. Because it relies on closed-form linear algebra rather than iterative loops, it can process large matrices orders of magnitude faster than likelihood-based alternatives. In benchmarks, $e$PCA processed a dataset in 14 seconds that required 3 hours for iterative methods.

Conclusion

$e$PCA provides a theoretically grounded and computationally efficient framework for dimensionality reduction in non-Gaussian settings. While it requires a specified noise distribution and manual rank selection, its ability to handle heteroskedastic noise makes it a powerful alternative to standard PCA for modern "big data" applications in imaging and biology. Future work may explore automated rank selection and noise model mixtures.

Summary updated

Abstract

Many applications involve large datasets with entries from exponential family distributions. Our main motivating application is photon-limited imaging, where we observe images with Poisson distributed pixels. We focus on X-ray Free Electron Lasers (XFEL), a quickly developing technology whose goal is to reconstruct molecular structure. In XFEL, estimating the principal components of the noiseless distribution is needed for denoising and for structure determination. However, the standard method, Principal Component Analysis (PCA), can be inefficient in non-Gaussian noise. Motivated by this application, we develop $e$PCA (exponential family PCA), a new methodology for PCA on exponential families. $e$PCA is a fast method that can be used very generally for dimension reduction and denoising of large data matrices with exponential family entries. We conduct a substantive XFEL data analysis using $e$PCA. We show that $e$PCA estimates the PCs of the distribution of images more accurately than PCA and alternatives. Importantly, it also leads to better denoising. We also provide theoretical justification for our estimator, including the convergence rate and the Marchenko–Pastur law in high dimensions. An open-source implementation is available.
BibTeX
@article{liu2018e,
  author = {Lydia T. Liu and Edgar Dobriban and Amit Singer},
  title = {{$e$PCA: High dimensional exponential family PCA}},
  journal = {The Annals of Applied Statistics},
  year = {2018},
  doi = {10.1214/18-aoas1146},
}

Loading related content...

Figure