Abstract
Recommendation models can effectively estimate underlying user interests and predict one's future behaviors by factorizing an observed user-item rating matrix into products of two sets of latent factors. However, the user-specific embedding factors can only be learned in a transductive way, making it difficult to handle new users on-the-fly. In this paper, we propose an inductive collaborative filtering framework that contains two representation models. The first model follows conventional matrix factorization which factorizes a group of key users' rating matrix to obtain meta latents. The second model resorts to attention-based structure learning that estimates hidden relations from query to key users and learns to leverage meta latents to inductively compute embeddings for query users via neural message passing. Our model enables inductive representation learning for users and meanwhile guarantees equivalent representation capacity as matrix factorization. Experiments demonstrate that our model achieves promising results for recommendation on few-shot users with limited training ratings and new unseen users which are commonly encountered in open-world recommender systems.
Introduction: The Transductive Bottleneck
Most modern recommendation engines rely on Collaborative Filtering (CF), specifically Matrix Factorization (MF). These models learn a unique "latent factor" (an embedding) for every user. While highly effective, they are fundamentally transductive: the model only knows about the users it saw during training.
When a new user enters the system—the "cold-start" or "open-world" problem—traditional MF models face a dilemma. They must either retrain the entire model from scratch, which is computationally expensive, or rely on side-features like age or location, which are often unavailable or poorly correlated with actual user interests.
Proposed Method: InDuctive Collaborative Filtering (IDCF)
The authors propose IDCF, a framework designed to be as expressive as Matrix Factorization while handling unseen users using only their initial interaction data. IDCF consists of two primary stages:
- Pretraining (The Meta Latents): The model first performs standard matrix factorization on a subset of "key users" to learn their latent embeddings, dubbed meta latents.
- Inductive Relation Model: For a new "query user," the model analyzes the items they have interacted with. It uses a multi-head attention mechanism to determine how this new user relates to the "key users."
- Neural Message Passing: The query user’s embedding is then calculated as a weighted sum of the key users' meta latents. Essentially, the model represents a new user as a combination of representative "anchor" users.
Crucially, the authors provide a theoretical proof (Theorem 1) showing that this inductive approach has the same representation capacity as standard, non-inductive Matrix Factorization.
Datasets and Evaluation
The model was evaluated on five diverse, real-world datasets: Douban, Movielens-100K, Movielens-1M (Explicit Feedback), and Amazon-Books and Amazon-Beauty (Implicit Feedback). These datasets were split to test two specific scenarios: Interpolation (handling users with very few ratings) and Extrapolation (handling entirely unseen users).
Key Insights and Results
- Representational Equivalence: The proof that inductive learning can match the power of MF is a significant finding. By treating key user embeddings as a basis, the model can reconstruct any user's preferences.
- Behavioral Proximity: A user's actions (what they rate) are far more informative than static attributes. IDCF maps users into a latent space based on behavioral similarity rather than metadata.
- Superiority in Cold-Start: IDCF consistently outperformed other inductive models (like NIMC and IGMC). On Movielens-1M, it achieved a 2.6% to 6.7% improvement in RMSE/NDCG for new users.
- Efficiency: Despite the added complexity of attention, IDCF scales linearly and is roughly 10x faster than graph-based competitors like IGMC.
- Robustness to Sparsity: In extreme cases where users have fewer than five historical ratings, IDCF-GC (the graph convolution variant) showed a 2.5% improvement over the best transductive models.
Conclusion
IDCF bridges the gap between the high performance of matrix factorization and the practical requirements of open-world recommendation. By enabling zero-shot user embedding generation through behavioral relations, it provides a scalable and effective solution for the cold-start problem. Future work may explore more sophisticated "key user" selection methods and extending the framework to handle new items.
Summary updated