Abstract
The Matrix Generalized Inverse Gaussian (MGIG) distribution is a versatile prior for symmetric positive definite matrices, yet its application has been hindered by computational difficulties. This paper establishes the unimodality of the MGIG distribution and demonstrates that its mode can be efficiently located by solving an Algebraic Riccati Equation (ARE). Leveraging this property, we introduce a mode-matched importance sampling strategy and a Collapsed Monte Carlo (CMC) algorithm for Probabilistic Matrix Factorization. Our approach significantly improves sampling efficiency and accuracy compared to traditional Markov Chain Monte Carlo (MCMC) methods.
Introduction
In Bayesian latent factor models, the Matrix Generalized Inverse Gaussian (MGIG) distribution serves as a flexible family of distributions that generalizes the Wishart and Inverse Wishart families. Despite its theoretical appeal, the MGIG distribution has been notoriously difficult to utilize. Prior to this work, fundamental properties such as unimodality were unproven, and efficient sampling techniques were lacking. Existing samplers often employed proposal distributions with modes poorly aligned with the target MGIG density, leading to vanishing effective sample sizes in high-dimensional settings.
Methods and Technical Contributions
Unimodality and the Algebraic Riccati Equation
A core contribution of this work is the proof that the MGIG distribution is indeed unimodal. We show that the problem of finding the distribution's mode—the most likely matrix—is equivalent to solving an Algebraic Riccati Equation (ARE). This connection is significant because AREs are central to optimal control theory, and robust, efficient numerical solvers are readily available. This transforms a difficult optimization problem into a well-understood linear algebra problem.
Mode-Matched Importance Sampling
To address the inefficiencies of "blind" sampling, we propose a mode-matched importance sampling strategy. By identifying the MGIG mode via the ARE, we can align the modes of Wishart or Inverse Wishart proposal distributions with the target MGIG density. This alignment ensures that samples are drawn from high-probability regions, drastically increasing the Effective Sample Size (ESS).
Collapsed Monte Carlo (CMC)
We apply these insights to Probabilistic Matrix Factorization (PMF), where the goal is to approximate a matrix $X$ as the product $UV^T$. By analytically marginalizing out one factor (e.g., $V$), we find that the posterior of the remaining factor ($U$) follows an MGIG distribution. This allows for a "collapsed" inference scheme where we sample directly from the marginal posterior, bypassing the slow convergence and high autocorrelation typical of standard Gibbs sampling or MCMC.
Experimental Results
The proposed CMC-PMF algorithm was evaluated on diverse datasets, including genetic SNP data, breast cancer gene expression, and the MovieLens 1M dataset.
- Accuracy: CMC-PMF consistently achieved lower log loss on held-out data compared to MCMC. In high-dimensional scenarios, MCMC often failed to capture the posterior density accurately, whereas CMC remained robust.
- Efficiency: The mode-matched sampler achieved an ESS of 550 per 1,000 samples, a substantial improvement over previous methods that yielded an ESS as low as 40.
- Computational Speed: On the SNP dataset, CMC reached convergence in approximately 75 seconds, while MCMC required over 12,000 seconds to achieve comparable results.
Conclusion
By bridging Bayesian statistics and control theory, this work transforms the MGIG distribution from a theoretical curiosity into a practical tool for high-dimensional inference. The discovery of the ARE-mode connection enables efficient, mode-matched sampling that outperforms traditional MCMC in both speed and accuracy. Future work will focus on scaling ARE solvers for extremely large-scale matrices and extending the CMC framework to more complex models like tensor factorization.
Summary updated