Metadata-dependent Infinite Poisson Factorization for Efficiently Modelling Sparse and Large Matrices in Recommendation

Trong Dinh Thac Do, Longbing CaoIJCAI 2018 · OpenReview

Abstract: Matrix Factorization (MF) is widely used in Recommender Systems (RSs) for estimating missing ratings in the rating matrix. MF faces major challenges of handling very sparse and large data. Poisson Factorization (PF) as an MF variant addresses these challenges with high efficiency by only computing on those non-missing elements. However, ignoring the missing elements in computation makes PF weak or incapable for dealing with columns or rows with very few observations (corresponding to sparse items or users). In this work, Metadata-dependent Poisson Factorization (MPF) is invented to address the user/item sparsity by integrating user/item metadata into PF. MPF adds the metadata-based observed entries to the factorized PF matrices. In addition, similar to MF, choosing the suitable number of latent components for PF is very expensive on very large datasets. Accordingly, we further extend MPF to Metadata-dependent Infinite Poisson Factorization (MIPF) that integrates Bayesian Nonparametric (BNP) technique to automatically tune the number of latent components. Our empirical results show that, by integrating metadata, MPF/MIPF significantly outperform the state-of-the-art PF models for sparse and large datasets. MIPF also effectively estimates the number of latent components.

Introduction

Recommender systems rely heavily on Collaborative Filtering, with Matrix Factorization (MF) being the dominant technique. However, real-world rating matrices, such as those from Netflix or Amazon, are massive and extremely sparse—often exceeding 98% empty. Standard MF is computationally expensive because it processes all elements, including missing ones.

Poisson Factorization (PF) addresses this efficiency bottleneck by computing only on non-missing data. However, because PF ignores missing elements, it performs poorly on rows or columns with very few observations, such as "cold-start" scenarios or highly sparse users and items. Furthermore, selecting the optimal number of latent components ($K$) for massive datasets typically requires training multiple models, creating a significant computational burden.

Methodology: MPF and MIPF

To address these limitations, this paper introduces two novel models that enrich the PF framework with metadata and automated parameter tuning.

Metadata-dependent Poisson Factorization (MPF)

MPF builds a hierarchical Gamma-Gamma-Poisson structure. Instead of relying solely on user-item interaction data, it incorporates user metadata (e.g., age, location) and item metadata (e.g., movie genre, book publisher) directly into the prior distributions of the latent factors. The weights of these metadata attributes influence the user's "latent behavior" and the item's "latent attractiveness," providing a robust fallback signal when interaction data is sparse.

Metadata-dependent Infinite Poisson Factorization (MIPF)

To avoid the expensive process of manually tuning $K$, MIPF extends MPF using Bayesian Nonparametrics (BNP)—specifically, a stick-breaking process. This allows the model to theoretically assume an infinite number of latent components, letting the data dictate the actual number of components needed during training.

Graphical models for (a) MPF and (b) MIPF, illustrating the integration of metadata and the hierarchical structure.

Crucially, both models utilize mean-field Variational Inference (VI) that retains the core advantage of PF: iterating only over non-zero ratings, which ensures high scalability.

Experimental Evaluation

The models were evaluated on four public datasets with varying levels of metadata richness: Movielens100K, Movielens1M, Movielens10M, and Book-Crossing. The Book-Crossing dataset provided the richest metadata, including detailed user demographics and book information (author, year, publisher).

Key Results and Discussion

Empirical results demonstrate that MPF and MIPF significantly outperform state-of-the-art PF baselines, including HPF, BNPPF, and HCPF, in terms of both Normalized Mean Precision and Recall.

Normalized Mean Precision (%) comparison across four datasets, showing the performance advantage of MPF and MIPF.

Impact of Metadata Richness

The models achieved the most substantial performance gains (up to 9%) on the Book-Crossing dataset, which correlates with its rich metadata. Conversely, improvements on Movielens10M were more modest, as it only contained "genre" metadata. This confirms that treating categorical metadata attributes as Gamma priors is highly effective for handling sparse data, allowing the model to group users or items by attributes when collaborative signals are weak.

Automated Tuning and Sparsity Handling

MIPF achieved accuracy on par with the best manually-tuned MPF models. This proves that MIPF can effectively estimate the optimal number of latent components in a single training run, saving massive amounts of hyperparameter tuning time. Qualitatively, MIPF successfully recommended a higher percentage of sparse items (those with <1% rating density) compared to baselines like HCPF, demonstrating its superior ability to handle the "long tail" of items.

Conclusion

By integrating metadata and Bayesian Nonparametrics into the Poisson Factorization framework, MPF and MIPF provide a scalable and accurate solution for large-scale, sparse recommendation tasks. While performance is dependent on metadata richness, the ability to automatically tune latent components makes MIPF a particularly powerful tool for practical applications where computational resources and data sparsity are primary concerns.

Summary updated

Abstract

Matrix Factorization (MF) is widely used in Recommender Systems (RSs) for estimating missing ratings in the rating matrix. MF faces major challenges of handling very sparse and large data. Poisson Factorization (PF) as an MF variant addresses these challenges with high efficiency by only computing on those non-missing elements. However, ignoring the missing elements in computation makes PF weak or incapable for dealing with columns or rows with very few observations (corresponding to sparse items or users). In this work, Metadata-dependent Poisson Factorization (MPF) is invented to address the user/item sparsity by integrating user/item metadata into PF. MPF adds the metadata-based observed entries to the factorized PF matrices. In addition, similar to MF, choosing the suitable number of latent components for PF is very expensive on very large datasets. Accordingly, we further extend MPF to Metadata-dependent Infinite Poisson Factorization (MIPF) that integrates Bayesian Nonparametric (BNP) technique to automatically tune the number of latent components. Our empirical results show that, by integrating metadata, MPF/MIPF significantly outperform the state-of-the-art PF models for sparse and large datasets. MIPF also effectively estimates the number of latent components.
BibTeX
@inproceedings{do2018metadata,
  author = {Trong Dinh Thac Do and Longbing Cao},
  title = {{Metadata-dependent Infinite Poisson Factorization for Efficiently Modelling Sparse and Large Matrices in Recommendation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  url = {https://openreview.net/forum?id=H1bxREz_WB},
}
Figure