Bayesian matrix factorization predicts how a user would rate an item. It does this by giving every user and every item a short vector of hidden traits, called latent factors, and taking their inner product. A prior distribution, meaning the modeler's assumption about what those trait vectors look like before seeing data, keeps the factors well behaved. For a popular movie with thousands of ratings, the data overwhelm the prior. For the many items in the long tail, the prior may be doing most of the work: over 30% of Netflix items have fewer than 10 ratings. Practitioners currently cannot tell which predictions are driven by data and which mainly restate the prior scale they picked. Testing this by refitting the model under several priors is expensive, and nobody does it item by item.
We propose to compute, for every item and every user, a prior-dependence score: the derivative of that entity's predictions with respect to the prior's hyperparameters. The computation reuses one variational fit and needs only a handful of iterative linear solves, which makes it feasible at Netflix scale. The decisive question is empirical. Do these scores tell us which recommendations actually change when the prior changes, and do they tell us more than the obvious cheap proxy, which is simply how many ratings an item has? If they do not, we will say so, and the cheap proxy wins.
Core Claim
Local prior sensitivity, computed with linear response variational Bayes, can be made scalable for matrix factorization by exploiting two structural facts. First, the number of prior hyperparameters is tiny, while the number of quantities we care about (per-item predictions) is huge. Second, the Hessian of the variational objective is sparse, so it can be applied to a vector cheaply. Linear response variational Bayes estimates how the fitted approximate posterior moves when the prior is nudged, by solving one linear system in the objective's Hessian.
Write $\eta$ for the variational parameters, $\alpha$ for the prior hyperparameters (the Laplace scales $b_u, b_v$, or the Gaussian variance $\sigma^2$), $\mathrm{KL}$ for the variational objective, and $H$ for its Hessian in $\eta$. The response of the fitted parameters is
$$ \frac{d\eta^*}{d\alpha} = -H^{-1}\,\frac{\partial^2 \mathrm{KL}}{\partial \eta\,\partial \alpha}, $$
so a predicted rating $r_{ij}=\mathbb{E}[u_i]^\top\mathbb{E}[v_j]$ has sensitivity $\nabla_\eta r_{ij}^\top \, d\eta^*/d\alpha$.
When $\alpha$ is low-dimensional, we need one conjugate-gradient solve per hyperparameter. Each solve uses only Hessian-vector products, costing roughly the number of observed ratings times $k^2$. Those few solves give sensitivities for all $m\times n$ predictions at once. When we instead perturb a per-item prior scale, a functional perturbation in the spirit of the stick-breaking work, a single adjoint solve gives the influence of every item's prior on any one global summary.
The second part of the claim concerns which quantities to measure. Under an isotropic Gaussian prior the posterior is unchanged by rotating the latent space, so $H$ is near-singular along those directions and factor-level sensitivities are meaningless. A Laplace prior breaks this rotational symmetry. As On the Geometry of Bayesian Graphical Models with Hidden Variables argues for latent-variable models generally, the prior then acts as a tie-breaker among equally good solutions. We therefore predict that factor interpretations under Laplace priors can be highly prior-sensitive even when predictions are stable. We will report sensitivities only for rotation-invariant quantities: predictions and top-N lists. We will measure factor-level fragility separately, as a finding rather than a nuisance.
Positioning
The method comes from Fast robustness quantification with variational Bayes. That paper introduced linear response sensitivity and showed it is both fast and revealing: it ran in 58 seconds versus 45 minutes of MCMC, and found that a 0.04 change in a prior covariance hyperparameter moved a posterior mean by 8.88 standard deviations. Its model, however, had a low-dimensional parameter vector and a dense Hessian. We replace the dense Hessian with matrix-free solves and replace a single scalar summary with per-entity scores.
Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics supplies the idea of influence functions over functional prior perturbations, which we adapt to per-item prior scales.
The models we analyze come from three papers. Sparse Probabilistic Matrix Factorization by Laplace Distribution for Collaborative Filtering introduced a Laplace-prior model that improves long-tail RMSE, but fixed $b_u, b_v$ by hand and gave no sensitivity analysis. Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching chooses hyperparameters by matching prior predictive moments, but offers no way to tell whether the chosen prior yields a fragile posterior. Covariate-moderated Empirical Bayes Matrix Factorization learns covariate-dependent priors by point optimization, with no characterization of how much the side information drives individual predictions.
All three set priors. None audits the result. Our scores are a post-fit diagnostic that can be applied to any of them.
The main limitation of our approach is constrained by the literature itself: linear response is local and says nothing about perturbations that jump between modes. We therefore treat the size of perturbation at which the linear prediction breaks down as a quantity to be measured, not assumed away.
Research Program
We will build variational inference for Gaussian-prior and Laplace-prior matrix factorization, using the Gaussian scale-mixture representation of the Laplace prior. On top of it we add an automatic-differentiation layer that supplies Hessian-vector products of the variational objective. We then implement forward-mode sensitivity for global hyperparameters and adjoint-mode sensitivity for per-entity prior scales.
Rotation directions are handled by restricting conjugate gradient to the subspace orthogonal to infinitesimal rotations of the factors, with a small damping term as a fallback. We will measure how much the damping changes the reported prediction sensitivities.
We will apply the tool to three settings, each a real use case:
- Laplace-prior model. Audit which long-tail items' recommendations depend on $b_v$.
- Prior-predictive-matched model on hetrec-lastfm. Audit whether the matched prior is fragile.
- Covariate-moderated priors on MovieLens 100K. Inject partially wrong genre labels and ask whether per-item sensitivity to the covariate-prior parameters flags the corrupted items.
Baselines
The ground-truth reference and main competitor is warm-started refitting. We rerun the same variational fit at perturbed hyperparameters, initialized from the original solution, and record how predictions change. This is the answer our derivatives try to predict. It is also what a skeptic would do instead, so we give it the same hardware and count its cost in wall-clock time and optimizer iterations. The derivative method beats it only if it matches refitting's per-item rankings closely (Spearman correlation above 0.9 for small perturbations) at substantially lower cost. Matching it is not enough when refits are cheap, so the comparison must also cover many simultaneous per-item perturbations, where refitting scales with the number of perturbed entities and the adjoint solve does not.
MCMC. On MovieLens 1M we also run Gibbs-sampled refits of the Laplace model, as in the sparse PMF paper. This checks that variational sensitivities track true posterior sensitivities, since mean-field variational inference is known to understate variance.
Naive reference. The do-nothing baseline is the item's rating count, the heuristic every practitioner already uses for "the prior dominates here."
Uncertainty baseline. We also compare against the item's mean-field posterior predictive variance. It is a different concept from prior sensitivity, but it is the uncertainty signal practitioners would reach for first.
Ablation. To isolate the claimed contribution of coupling through shared factors, we compute a block-diagonal approximation that ignores coupling between users and items and inverts each item's own small block. If this ablation matches the full solve, the coupling claim fails.
For predictive accuracy context, the strongest reported number on Netflix is SPMF RMSE of about 0.8809 at 150 latent features, against 0.9125 for BPMF, both from the sparse PMF paper. Our variational reimplementation must land near this before its sensitivities are worth trusting. We found no reported result on Lacuna for prior-sensitivity accuracy in matrix factorization.
Why not just…?
Why not just use rating counts? This is the real threat. Sensitivity may scale roughly like one over the count, in which case the method adds nothing. We will test it directly by computing the partial rank correlation between derivative scores and actual refit changes, controlling for item count. The test is sharpest for items with equal counts whose raters differ: raters who are themselves sparsely observed should make an item more prior-dependent. If the partial correlation is near zero, we will report that counts suffice for global hyperparameters. We would then retain only the per-item functional and covariate-corruption analyses, where counts have no obvious analogue.
Why not just tune hyperparameters by cross-validation or marginal likelihood? Tuning picks a single hyperparameter setting. It does not say which predictions would change under a nearby equally plausible setting. Empirical Bayes in non-convex factor models also inherits sensitivity to initialization, as the cEBMF paper notes, and broad priors in high dimensions concentrate mass at the boundaries of the parameter space, as shown in The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?. We will run sensitivity at the cross-validated and prior-predictive-matched settings, so tuning and auditing are complements in our experiments, not rivals.
Why not just drop priors and use nuclear-norm regularization? Learning Preferences from Assortment Choices in a Heterogeneous Population shows that nuclear-norm estimation has good sample complexity. But the regularization weight plays exactly the role of a prior scale, so the question "which predictions depend on it" remains. Our derivative applies equally to the penalized point estimate, and we will report it for that case as well.
Proposed Experiments
Validation on MovieLens 100K. We first check derivative scores against warm-started refits and MCMC refits on MovieLens 100K, using a subsampled model small enough to check convergence of the iterative solves. We perturb $b_v$ and $\sigma^2$ by 1%, 10%, 50%, and 200%, and plot prediction error against perturbation size to locate where the linear approximation fails.
Scale test. We then repeat the comparison on MovieLens 1M and Netflix, measuring top-10 list overlap (Jaccard) predicted from first-order changes versus observed after refitting, bucketed by item popularity. Throughout, we report the partial correlation over rating count described above. That number decides whether the method is worth using.
Prior-family swap. We swap the prior family entirely, from Gaussian to Laplace, using a functional perturbation path. This tests the predicted split between stable predictions and unstable factor directions, measured as subspace angles between fitted item-factor matrices.
Fragility of prior-predictive matching. On hetrec-lastfm we ask whether the prior-predictive-matched hyperparameters sit in a high-sensitivity region compared with nearby settings.
Corrupted side information. On MovieLens 100K with cEBMF, we flip genre labels for a known 10% of items and measure how well per-item sensitivity to the covariate-prior parameters recovers the corrupted set (area under the ROC curve). Count and posterior variance serve as competing detectors.
The project succeeds if derivative scores match refits at a fraction of the cost and predict fragile recommendations beyond what rating counts predict. It fails informatively if counts explain everything.
References
- On the Geometry of Bayesian Graphical Models with Hidden Variables
- Fast robustness quantification with variational Bayes
- Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics
- Sparse Probabilistic Matrix Factorization by Laplace Distribution for Collaborative Filtering
- Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching
- Covariate-moderated Empirical Bayes Matrix Factorization
- The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?
- Revealed Preference at Scale: Learning Personalized Preferences from Assortment Choices