Gamma-Poisson Dynamic Matrix Factorization Embedded with Metadata Influence

Trong Dinh Thac Do, Longbing CaoNeurIPS 2018 · OpenReview

TL;DR

The mGDMF model tackles the "triple threat" of modern recommendation systems: massive scale, extreme sparsity, and evolving user preferences. By combining a Gamma-Poisson factorization structure with metadata integration and Gamma-Markov chains, it provides a scalable way to make accurate recommendations even for "cold-start" users and items.

What Problem Does This Paper Address?

Traditional Matrix Factorization (MF) models, while powerful, often stumble when faced with real-world data. The authors identify three critical gaps in existing recommendation systems:

  1. Sparsity and Cold-Starts: Most users only rate a tiny fraction of available items. When a new user or item enters the system (the "cold-start" problem), standard MF models have no historical data to work with.
  2. Dynamics: User tastes aren't static; a child who loves cartoons grows into an adult who prefers documentaries. Many models incorrectly assume preferences are fixed over time.
  3. Scalability: Modern datasets like Netflix contain millions of ratings. Processing these globally is computationally prohibitive.

While previous Poisson-based models addressed sparsity and scale, they often ignored rich metadata (like user age or movie genre) or failed to model how preferences drift smoothly over time.

The mGDMF Approach

The authors propose mGDMF, a conjugate Gamma-Poisson model that integrates three distinct strategies to overcome these challenges:

  • Gamma-Poisson Structure: Unlike Gaussian-based MF, the Poisson distribution is naturally suited for sparse, non-negative count data (like ratings or play counts). It handles the "long-tail" distribution of real-world data much more effectively.
  • Metadata Integration: The model incorporates observable attributes (e.g., a user's location or a movie's release year) as Gamma priors for latent features. This allows the model to "guess" the preferences of a user with zero ratings based on their demographic similarity to others.
  • Gamma-Markov Chains: To capture temporal changes, the model uses a chain of latent factors that "drift" over time. They introduce auxiliary variables to ensure these transitions are smooth and mathematically conjugate, preventing the instability often seen in dynamic models.
  • Stochastic Variational Inference (SVI): To handle massive datasets, the authors use SVI. This allows the model to learn from small, random batches of data rather than the entire dataset at once, making it computationally efficient and scalable.

Key Insights and Ideas

  • Metadata as a Structural Bridge: Metadata isn't just "extra info"—it acts as a prior that weights the latent features. This creates a "coupling" effect where users with similar metadata share latent preferences, which is the key to solving the cold-start problem.
  • Balancing Static and Dynamic Factors: The model decomposes preferences into a "global static" portion (long-term personality) and a "local dynamic" portion (current phase of life). This prevents the model from overreacting to short-term noise while still allowing for genuine taste evolution.
  • The Power of Conjugacy: By ensuring the entire model uses conjugate distributions (Gamma-Gamma, Gamma-Poisson), the authors derived closed-form update equations. This makes the inference process significantly faster and more stable than models requiring complex sampling or optimization.

Key Results

The model was evaluated on five large-scale dynamic datasets: Netflix-Time, Netflix-Full, Yelp-Active, LFM-Tracks, and LFM-Bands. Across the board, the mGDMF model consistently outperformed state-of-the-art static and dynamic models (like dPF and DCPF).

Precision and Recall at 50 for mGDMF compared to baseline models across multiple datasets.
  • Accuracy Gains: On the "Netflix-Full" dataset, which is notoriously long-tailed, mGDMF showed a 10% improvement in recommendation precision.
  • Cold-Start Performance: In scenarios with 0% rating density (pure cold-start), mGDMF could still recommend relevant items with 0.5% precision, whereas competing dynamic models (like DCPF) failed entirely.
  • Ranking Quality: The model achieved massive gains in Normalized Discounted Cumulative Gain (NDCG) and Area Under the Curve (AUC).
NDCG and AUC performance comparison showing mGDMF outperforming state-of-the-art baselines.

As shown in the performance tables, mGDMF yielded NDCG improvements ranging from 23% to 240% over baselines, particularly shining on the metadata-rich Yelp dataset.

Limitations and Future Work

While mGDMF is highly effective for categorical and binary metadata, the paper does not deeply explore how to integrate unstructured data like raw text reviews or images. The authors suggest that future iterations could tackle even more complex challenges, such as:

  • Context-Aware Chatbots: Using the dynamic metadata influence to power conversational agents.
  • Deep Integration: Combining this probabilistic framework with deep learning to capture more complex, non-linear relationships in textual and sequential information.

Summary updated

Abstract

A conjugate Gamma-Poisson model for Dynamic Matrix Factorization incorporated with metadata influence (mGDMF for short) is proposed to effectively and efficiently model massive, sparse and dynamic data in recommendations. Modeling recommendation problems with a massive number of ratings and very sparse or even no ratings on some users/items in a dynamic setting is very demanding and poses critical challenges to well-studied matrix factorization models due to the large-scale, sparse and dynamic nature of the data. Our proposed mGDMF tackles these challenges by introducing three strategies: (1) constructing a stable Gamma-Markov chain model that smoothly drifts over time by combining both static and dynamic latent features of data; (2) incorporating the user/item metadata into the model to tackle sparse ratings; and (3) undertaking stochastic variational inference to efficiently handle massive data. mGDMF is conjugate, dynamic and scalable. Experiments show that mGDMF significantly (both effectively and efficiently) outperforms the state-of-the-art static and dynamic models on large, sparse and dynamic data.
BibTeX
@inproceedings{do2018gamma,
  author = {Trong Dinh Thac Do and Longbing Cao},
  title = {{Gamma-Poisson Dynamic Matrix Factorization Embedded with Metadata Influence}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=HyZmDOWu-r},
}
Figure