Bayesian Uncertainty Quantification for Low-Rank Matrix Completion

Henry Shaowu Yuchi, Simon Mak, Yao XieBayesian 2022 · DOI

Abstract

We consider the problem of uncertainty quantification for an unknown low-rank matrix X, given a partial and noisy observation of its entries. This quantification of uncertainty is essential for many real-world problems, including image processing, satellite imaging, and seismology, providing a principled framework for validating scientific conclusions and guiding decision-making. However, existing literature has mainly focused on the completion (i.e., point estimation) of the matrix X, with little work on investigating its uncertainty. To this end, we propose in this work a new Bayesian modeling framework, called BayeSMG, which parametrizes the unknown X via its underlying row and column subspaces. This Bayesian subspace parametrization enables efficient posterior inference on matrix subspaces, which represents interpretable phenomena in many applications. This can then be leveraged for improved matrix recovery. We demonstrate the effectiveness of BayeSMG over existing Bayesian matrix recovery methods in numerical experiments, image inpainting, and a seismic sensor network application.

Introduction

Matrix completion is a ubiquitous challenge in modern data science, powering applications from recommendation systems to satellite image restoration. While significant progress has been made in point estimation—finding the single most likely value for a missing entry—the field has largely overlooked Uncertainty Quantification (UQ). In high-stakes domains like seismology or medical imaging, a point estimate alone is insufficient; practitioners require reliable "error bars" to assess the trustworthiness of the recovered data. Existing Bayesian methods often struggle with UQ because they fail to explicitly account for the geometric subspace structure inherent in low-rank matrices, frequently resulting in overconfident or inaccurate predictions.

Methodology: The BayeSMG Framework

The core contribution of this work is BayeSMG (Bayesian Singular Matrix-variate Gaussian). Unlike traditional approaches that treat matrix entries as independent, BayeSMG parametrizes the unknown matrix $X$ through its Singular Value Decomposition (SVD): $X = UDV^T$.

Key technical components include:

  • Subspace Priors: The model utilizes the matrix von Mises-Fisher (vMF) distribution to place priors on the row and column subspaces ($U$ and $V$). This provides a mathematically principled way to model data on a Stiefel manifold (the space of orthonormal matrices).
  • Hierarchical Modeling: By treating singular values and noise variances as part of a hierarchical structure, the model adaptively learns noise levels from the observed data.
  • Efficient Posterior Inference: A specialized Gibbs sampler alternates between imputing missing values and updating subspace parameters, enabling efficient inference for large-scale matrices.
  • Rank Estimation: The framework employs a Maximum A Posteriori (MAP) approach linked to nuclear-norm regularization to estimate the matrix rank prior to full Bayesian analysis.

Experimental Evaluation and Results

The authors evaluated BayeSMG across synthetic data, image inpainting (including solar flare and high-resolution campus imagery), and real-world seismic sensor data from Yellowstone National Park.

Key Results

  • Superior Coverage: In numerical simulations, BayeSMG demonstrated significantly more reliable uncertainty intervals. While a standard "plug-in" approach achieved only a ~44-46% coverage ratio for 95% confidence intervals, BayeSMG achieved approximately 94-95%, offering a far more honest assessment of estimation error.
Comparison of coverage ratios between a standard plug-in approach (left) and BayeSMG (right), demonstrating the latter's superior uncertainty quantification.
  • Enhanced Recovery: On solar flare satellite imagery, BayeSMG achieved a Mean Frobenius Error (MFE) of 31.0, vastly outperforming the Bayesian Probabilistic Matrix Factorization (BPMF) method (MFE of 350.8), which suffered from significant "blocking" artifacts.
  • Scalability and Precision: The algorithm successfully processed a $1911 \times 3000$ image in roughly two hours on standard hardware. In seismic applications, it provided narrower uncertainty intervals than competing methods, facilitating more confident identification of sensor correlations.

Discussion and Insights

Directly modeling the underlying geometry allows BayeSMG to capture global matrix structures, making it particularly robust when entire rows or columns are missing. The research also establishes a theoretical link between matrix coherence and uncertainty: the model naturally assigns higher uncertainty to "spikier" regions where information is more localized and harder to recover. Furthermore, the authors prove "error monotonicity," ensuring that expected uncertainty decreases as more noisy observations are collected.

Limitations and Future Work

Current limitations include the use of a "plug-in" MAP estimate for rank rather than a fully Bayesian treatment, which remains computationally prohibitive. Additionally, the model assumes i.i.d. Gaussian noise, whereas real-world sensor data often exhibits spatially correlated errors. Future research will explore these areas and investigate the use of BayeSMG's UQ to guide active learning and experimental design.

Summary updated

Abstract

We consider the problem of uncertainty quantification for an unknown low-rank matrix X, given a partial and noisy observation of its entries. This quantification of uncertainty is essential for many real-world problems, including image processing, satellite imaging, and seismology, providing a principled framework for validating scientific conclusions and guiding decision-making. However, existing literature has mainly focused on the completion (i.e., point estimation) of the matrix X, with little work on investigating its uncertainty. To this end, we propose in this work a new Bayesian modeling framework, called BayeSMG, which parametrizes the unknown X via its underlying row and column subspaces. This Bayesian subspace parametrization enables efficient posterior inference on matrix subspaces, which represents interpretable phenomena in many applications. This can then be leveraged for improved matrix recovery. We demonstrate the effectiveness of BayeSMG over existing Bayesian matrix recovery methods in numerical experiments, image inpainting, and a seismic sensor network application.
BibTeX
@inproceedings{yuchi2022bayesian,
  author = {Henry Shaowu Yuchi and Simon Mak and Yao Xie},
  title = {{Bayesian Uncertainty Quantification for Low-Rank Matrix Completion}},
  booktitle = {Bayesian},
  year = {2022},
  doi = {10.1214/22-ba1317},
}
Figure