Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach

Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Junchi Yan, Hongyuan Zha

TL;DR

The InDuctive Collaborative Filtering (IDCF) framework bridges the gap between high-performance matrix factorization and the need for flexible, "open-world" recommendation. By learning to synthesize new user embeddings from a set of pretrained "meta latents" via an attention-based relation model, IDCF handles unseen users on-the-fly without sacrificing the expressive power of traditional collaborative filtering.

The Challenge of Open-World Recommendation

Most modern recommender systems rely on Collaborative Filtering (CF), typically implemented via Matrix Factorization (MF). While MF is incredibly expressive—capturing nuanced user preferences by learning specific "latent factors" (embeddings)—it is fundamentally transductive. This means the model can only make predictions for users it "saw" during training. When a new user arrives (the "open-world" scenario), the system must either be retrained from scratch or rely on "feature-driven" models that use side information like age or occupation.

However, high-quality user features are often unavailable due to privacy concerns, and simple feature-driven models often lack the "expressive capacity" of MF. The authors tackle a difficult question: Can we build a model that is as expressive as matrix factorization but can also handle new users inductively?

Comparison of traditional CF, feature-driven models, and the proposed IDCF.

The IDCF Framework

The authors propose IDCF, a two-stage framework that treats user representation as a message-passing problem on a latent graph:

  1. Pretraining (Meta Latents): The model first performs standard matrix factorization on a group of "key users" to obtain their embeddings, which the authors call meta latents.
  2. Inductive Relation Model: Instead of learning a new embedding for every new user, IDCF learns a relation model (using a multi-head attention mechanism). This model looks at a new "query user’s" limited rating history and determines their relationship to the key users.
  3. Neural Message Passing: The query user’s embedding is then inductively computed as a weighted combination of the key users' meta latents.

The framework supports two main scenarios: Interpolation (handling few-shot users with limited ratings) and Extrapolation (handling entirely unseen zero-shot users). For extrapolation, they introduce a self-supervised contrastive loss to ensure the inductively generated embeddings remain consistent with the latent space.

The IDCF workflow for inductive learning and extrapolation.

Datasets and Evaluation

The model was evaluated on five diverse recommendation benchmarks to test both "few-shot" performance (users with very few training ratings) and "new user" performance (users completely excluded from the training set):

  • Explicit Feedback (Ratings 1-5): Douban, Movielens-100K (ML-100K), and Movielens-1M (ML-1M).
  • Implicit Feedback (Interactions): Amazon-Books and Amazon-Beauty.

Key Insights and Ideas

  • Equivalent Capacity: Theoretically, the authors prove that IDCF can minimize reconstruction loss to the same level as vanilla matrix factorization. This means you don't lose accuracy by switching to an inductive approach.
  • The Power of "Key Users": The model identifies that some key users are more "informative" than others. Users with more historical ratings tend to serve as better anchors for synthesizing new user representations.
  • Feature-Free Induction: Unlike many inductive models that require user profiles (age, gender, etc.), IDCF achieves induction using only the user-item interaction matrix.
  • Scalability via Sampling: To handle large datasets, the authors use a sampling strategy for the attention mechanism, ensuring the model scales linearly rather than quadratically with the number of users.

Key Results

IDCF consistently outperformed existing inductive baselines (like NIMC and IGMC) and matched or exceeded the performance of state-of-the-art transductive models:

  • Few-Shot Performance: On ML-1M, IDCF-GC achieved a 2.5% improvement in RMSE over the best transductive GNN models for users with fewer than five ratings.
  • New User Extrapolation: On the Douban dataset, IDCF improved RMSE by 2.6% and NDCG by 3.6% compared to the best existing inductive competitors.
  • Implicit Feedback: On Amazon-Books, IDCF showed a massive jump in performance for new users, outperforming item-based CF models (like FISM) by over 25% in AUC.

Limitations and Future Work

While IDCF is more efficient than retraining a full MF model, it is still computationally heavier than a simple forward pass in a feature-driven model. The authors note that IDCF spends about twice as much time per epoch as standard GNN-based CF models.

Future work could explore more sophisticated ways to select "key users" to further optimize the meta-latent space. Additionally, while the paper focuses on user-item matrices, the authors suggest this "meta-latent" paradigm could be generalized to other entities in the wild, potentially opening new doors for general representation learning.

Summary updated

Abstract

Recommendation models can effectively estimate underlying user interests and predict one’s future behaviors by factorizing an observed user-item rating matrix into products of two sets of latent fa...
BibTeX
@inproceedings{wu2021towards,
  author = {Qitian Wu and Hengrui Zhang and Xiaofeng Gao and Junchi Yan and Hongyuan Zha},
  title = {{Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=-0p9w3D8GT},
}
Figure