Bayesian Probabilistic Multi-Topic Matrix Factorization for Rating Prediction

Keqiang Wang, Wayne Xin Zhao, Hongwei Peng, Xiaoling WangIJCAI 2016

Introduction

In the field of recommender systems, Matrix Factorization (MF) has long been the standard for predicting user ratings. By projecting users and items into a shared latent space, MF captures global patterns of preference. However, this "one-size-fits-all" approach often overlooks the nuanced, local nature of human interests. A user might exhibit very different rating behaviors when evaluating a high-octane action movie versus a slow-burn period drama.

While Local Matrix Factorization (LMF) attempts to address this by partitioning the rating matrix into sub-matrices or clusters, many existing methods rely on rigid, heuristic clustering techniques like k-means. These methods often fail to capture the overlapping interests of users and lack a rigorous probabilistic foundation. This paper introduces Bayesian Probabilistic Multi-Topic Matrix Factorization (BPMTMF), a model designed to bridge the gap between the semantic richness of topic modeling and the predictive precision of local matrix factorization.

The BPMTMF Framework

BPMTMF integrates topic modeling and matrix factorization into a unified Bayesian generative framework. The core innovation lies in treating a user's rating history as a "document" and the items they have interacted with as "words."

Key Components:

  1. Topic Modeling for Items: Using a structure inspired by Latent Dirichlet Allocation (LDA), the model discovers $K$ latent topics, which represent multinomial distributions over items. Each user is assigned a topical distribution ($\theta_u$), allowing for "soft" clustering where a user can belong to multiple interest groups simultaneously.
  2. Topic-Specific Latent Spaces: Unlike standard MF, BPMTMF assigns a unique latent vector to every user and item for each specific topic ($P_u^{(k)}$ and $Q_m^{(k)}$). This allows the model to learn how a user's preferences shift depending on the context of the topic.
  3. Bayesian Inference: To ensure robustness and prevent overfitting, the model employs Gaussian-Wishart priors. Inference is performed using a Collapsed Gibbs Sampler, which iteratively updates topic assignments and latent vectors, making the model less sensitive to hyper-parameter tuning than its non-Bayesian counterparts.

Experimental Results

The authors evaluated BPMTMF on two industry-standard benchmarks: the MovieLens 10M dataset and the Netflix dataset. These datasets provide a rigorous test for any recommendation algorithm due to their extreme sparsity and massive scale.

Quantitative Performance

BPMTMF demonstrated superior accuracy compared to state-of-the-art local MF baselines, including LLORMA and WEMAREC. By leveraging the ensemble of topic-specific predictions weighted by the learned topical distributions, the model achieved significantly lower Root Mean Square Error (RMSE) values.

RMSE performance comparison across different methods on MovieLens and Netflix datasets.

As shown in the results table, BPMTMF consistently outperformed all competing methods. On the MovieLens dataset, it achieved an RMSE of 0.7679, a notable improvement over LLORMA's 0.7834. This performance gain is attributed to the model's ability to capture complex, overlapping local structures that global or hard-clustered models miss.

Qualitative Analysis and Interpretability

One of the primary advantages of BPMTMF is its semantic interpretability. Because the local clusters are derived from a topic model, they correspond to coherent, human-readable categories.

Qualitative comparison of top items for Action and Drama topics between LLORMA and BPMTMF.

When examining the "top ten" lists for specific topics, BPMTMF produces highly consistent results. For example, in the "Action" topic, it correctly identifies genre staples like Raiders of the Lost Ark and Die Hard. In contrast, heuristic methods like LLORMA often include semantically unrelated titles (e.g., American Beauty or Shakespeare in Love) within an action-oriented cluster. This demonstrates that BPMTMF's latent spaces are capturing genuine thematic structures rather than just statistical noise.

Conclusion and Future Work

BPMTMF offers a principled, Bayesian approach to local matrix factorization that improves both prediction accuracy and model transparency. While the model is more computationally intensive than standard MF—with a complexity of $O(DKS)$ per iteration—the gains in performance and interpretability make it a compelling choice for large-scale recommendation tasks.

Future research directions include addressing the "cold start" problem by incorporating item metadata (such as plot summaries or tags) directly into the topic modeling component, as well as exploring more efficient inference techniques to reduce the computational overhead of the Gibbs sampler.

Summary updated

BibTeX
@inproceedings{wang2016bayesian,
  author = {Keqiang Wang and Wayne Xin Zhao and Hongwei Peng and Xiaoling Wang},
  title = {{Bayesian Probabilistic Multi-Topic Matrix Factorization for Rating Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure