Abstract
Recommendation models can effectively estimate underlying user interests and predict one's future behaviors by factorizing an observed user-item rating matrix into products of two sets of latent factors. However, the user-specific embedding factors can only be learned in a transductive way, making it difficult to handle new users on-the-fly. In this paper, we propose an inductive collaborative filtering framework that contains two representation models. The first model follows conventional matrix factorization which factorizes a group of key users' rating matrix to obtain meta latents. The second model resorts to attention-based structure learning that estimates hidden relations from query to key users and learns to leverage meta latents to inductively compute embeddings for query users via neural message passing. Our model enables inductive representation learning for users and meanwhile guarantees equivalent representation capacity as matrix factorization. Experiments demonstrate that our model achieves promising results for recommendation on few-shot users with limited training ratings and new unseen users which are commonly encountered in open-world recommender systems.
Introduction
Collaborative Filtering (CF), particularly Matrix Factorization (MF), has long been the gold standard for personalized recommendation. By decomposing a user-item interaction matrix into low-dimensional latent factors, MF captures the nuanced preferences of users. However, MF is fundamentally transductive: it assigns a unique, static embedding to every user ID present during training. In modern "open-world" systems where new users arrive continuously, this creates a significant bottleneck. Standard MF cannot generate predictions for these new users without a full model retrain, which is computationally prohibitive for real-time deployment.
The Transductive Bottleneck in Collaborative Filtering
Existing attempts to bridge this gap often fall short. Feature-driven models attempt to predict user preferences using side information like demographics, but such data is frequently sparse, noisy, or unavailable due to privacy concerns. Heuristic methods like K-Nearest Neighbors (KNN) are inductive but lack the representational depth of latent factor models. While Graph Neural Networks (GNNs) offer an inductive alternative, they often struggle with scalability or fail to provide explicit user representations suitable for diverse downstream tasks.
The IDCF Framework
To address these challenges, we propose InDuctive Collaborative Filtering (IDCF). The core innovation of IDCF is the conceptualization of users as combinations of "basis vectors" or archetypes. The framework partitions the user base into two sets: Key Users (the anchors) and Query Users (new or few-shot users).
The architecture operates in two primary stages:
- Meta-Latent Learning: The model performs standard matrix factorization on the Key Users to derive high-quality, user-specific embeddings, termed "meta-latents."
- Inductive Relation Modeling: For Query Users, the model employs a multi-head attention mechanism. By analyzing a Query User’s sparse interaction history, the model determines their "hidden relations" to the Key Users.
- Neural Message Passing: These learned relations serve as weights for a message-passing process, where information from the well-established meta-latents is propagated to synthesize an embedding for the Query User.
Theoretical Capacity and Training
A critical contribution of this work is the theoretical proof (Theorem 1) that this inductive approach maintains the same representation capacity as traditional transductive MF. This ensures that moving to an inductive framework does not sacrifice the model's ability to capture complex user behaviors. Furthermore, for the "extrapolation" case—where users have no prior history—we introduce a self-supervised contrastive loss to ensure that synthesized embeddings remain consistent with the global latent space.
Experimental Evaluation
We evaluated IDCF across five real-world benchmarks, including Movielens (100K and 1M) and large-scale Amazon datasets (Books and Beauty). The results demonstrate that IDCF-GC significantly outperforms state-of-the-art inductive models like NIMC and BOMIC. In extrapolation tests, it improved RMSE by up to 4.0% on Movielens-1M and AUC by 25.7% on Amazon-Books.
Crucially, IDCF matches the performance of transductive MF models on seen users while offering a 10x speedup per epoch compared to advanced graph-based inductive models like IGMC. This efficiency stems from a sampling strategy for Key Users that keeps training time linear relative to dataset size.
Conclusion
IDCF provides a scalable and theoretically grounded solution to the open-world recommendation problem. By leveraging meta-latents and attention-based relation learning, it combines the expressiveness of matrix factorization with the flexibility of inductive learning. Future work will explore more sophisticated key user selection strategies and the potential for cross-domain transfer learning using the meta-latent paradigm.
Summary updated