Recommending Burgers based on Pizza Preferences: Addressing Data Sparsity with a Product of Experts

Martin Milenkoski, Diego Antognini, Claudiu MusatarXiv 2021 · OpenReview

TL;DR

This paper introduces a "Product of Experts" (POE) model to address data sparsity in recommendation systems by leveraging user preferences across multiple domains. Their approach effectively uses knowledge from a user's interactions in one domain (e.g., pizza) to make recommendations in another domain where they have little to no history (e.g., burgers), even outperforming single-domain models in some scenarios.

What Problem Does This Paper Address?

Recommender systems thrive on user interaction data. However, a common challenge is "data sparsity," where users have limited or no interaction history in a specific domain. This "cold-start" problem makes it difficult to provide personalized recommendations. Traditional methods often struggle when users have extensive history in one area but are new to another. This paper tackles this by exploring how user preferences in a "source" domain can inform recommendations in a "target" domain, even when the target domain has very little data.

Methods

The core of the proposed solution is a Product of Experts (POE) architecture built upon Variational Autoencoders (VAEs).

  • VAE Foundation: They start with VAEs, a popular deep learning model for collaborative filtering, which learns latent representations of users and items.
  • Multi-Domain Expansion: The key innovation is extending the VAE framework to handle multiple domains simultaneously. They assume that user preferences across different domains are conditionally independent given a common latent representation.
  • Product of Experts: Instead of a single model for all domains, they model the joint distribution of user preferences across domains as a "product of experts." Each "expert" is a VAE trained on a specific domain. This allows the model to be resilient to missing data in one or more domains. If a user has no data for a particular domain, that domain's "expert" can be effectively ignored during inference.
  • Weakly-Supervised Training: The POE model is designed to be trained even when users have interactions in only a subset of the domains, making it suitable for real-world scenarios.

Datasets

The authors evaluate their POE model on publicly available datasets from:

  • Amazon Reviews: They use data from popular categories like Books, Kindle Store, Movies, and Clothing, selecting combinations to maximize user overlap and study semantic similarity (e.g., Books and Kindle).
  • Yelp Businesses: They use data from categories such as Restaurants, Hotels, Shopping, Food, Burgers, and Pizza, again focusing on combinations to explore different levels of domain similarity.

For evaluation, they binarize ratings (≥ 3.5 indicating a positive interaction) and filter users with fewer than 5 reviews. They split users into training (95%) and testing (5%) sets, with a portion of the test set interactions masked for evaluation.

Key Insights and Ideas

  • Cross-Domain Synergy: The fundamental idea is that understanding a user's taste in one area (like their preference for certain pizza toppings) can reveal underlying preferences that are transferable to another area (like their preference for burger types).
  • Handling Missing Data Gracefully: The POE architecture's ability to "ignore" domains with no user data is crucial. This makes it practical for real-world applications where complete user profiles across all possible domains are rare.
  • Surprising Performance: In some cross-domain scenarios, the POE model, even when not using any target domain data during evaluation, achieved performance comparable to or even better than a standard VAE trained only on the target domain. This highlights the power of transferring knowledge.
  • Domain Weighting Matters: The paper shows that assigning different weights to different domains during training can significantly impact performance, suggesting that not all domain combinations contribute equally.

Key Results

  • Single-Domain Improvement: In single-domain recommendation settings, the POE model often outperforms traditional single-domain VAEs, especially when trained on multiple domains simultaneously.
  • Cross-Domain Competitiveness: For cross-domain recommendations, the POE model consistently outperforms a simple "most popular items" baseline. Crucially, it achieves performance close to, and sometimes exceeding, a single-domain VAE trained on the target domain, demonstrating effective knowledge transfer.
  • Pareto Fronts: The results are visualized using Pareto fronts, showing trade-offs between performance metrics (Recall@50, NDCG@50) across different domains. These plots illustrate how the POE model can achieve good performance across multiple domains simultaneously or excel in one domain at the expense of another, depending on the learned weights.

Limitations and Future Work

  • Hyperparameter Tuning: The authors note that domain weights significantly impact performance and suggest treating them as hyperparameters to be tuned using standard methods or learned jointly during training.
  • Understanding Domain Correlations: Further research is needed to understand why certain domain combinations work better than others, potentially using multi-task learning techniques to measure domain correlations.
  • Visualizing Latent Space: Analyzing the learned latent distributions through similarity measures or dimensionality reduction techniques could provide deeper insights.
  • Generalization to More Domains: While the POE approach is designed to extend to more than two domains, the paper focuses on pairwise combinations. Further exploration of larger multi-domain settings would be valuable.

Summary updated

Abstract

In this paper, we describe a method to tackle data sparsity and create recommendations in domains with limited knowledge about user preferences. We expand the variational autoencoder collaborative filtering from a single-domain to a multi-domain setting. The intuition is that user-item interactions in a source domain can augment the recommendation quality in a target domain. The intuition can be taken to its extreme, where, in a cross-domain setup, the user history in a source domain is enough to generate high-quality recommendations in a target one. We thus create a Product-of-Experts (POE) architecture for recommendations that jointly models user-item interactions across multiple domains. The method is resilient to missing data for one or more of the domains, which is a situation often found in real life. We present results on two widely-used datasets - Amazon and Yelp, which support the claim that holistic user preference knowledge leads to better recommendations. Surprisingly, we find that in some cases, a POE recommender that does not access the target domain user representation can surpass a strong VAE recommender baseline trained on the target domain.
BibTeX
@misc{milenkoski2021recommending,
  author = {Martin Milenkoski and Diego Antognini and Claudiu Musat},
  title = {{Recommending Burgers based on Pizza Preferences: Addressing Data Sparsity with a Product of Experts}},
  year = {2021},
  eprint = {2104.12822},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.12822},
}
Figure