Coupled Variational Recurrent Collaborative Filtering

Qingquan Song, Shiyu Chang, Xia Hu

Abstract

We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Although deep neural networks have demonstrated the effectiveness of recommendation tasks, it is lack of explorations on integrating probabilistic models and deep architectures under streaming recommendation settings. Conjoining the complementary advantages of probabilistic models and deep neural networks could enhance both model effectiveness and the understanding of inference uncertainties. To bridge the gap, in this paper, we propose a Coupled Variational Recurrent Collaborative Filtering (CVRCF) framework based on the idea of Deep Bayesian Learning to handle the streaming recommendation problem. The framework jointly combines stochastic processes and deep factorization models under a Bayesian paradigm to model the generation and evolution of users' preferences and items' popularities. To ensure efficient optimization and streaming update, we further propose a sequential variational inference algorithm based on a cross variational recurrent neural network structure. Experimental results on three benchmark datasets demonstrate that the proposed framework performs favorably against the state-of-the-art methods in terms of both temporal dependency modeling and predictive accuracy. The learned latent variables also provide visualized interpretations for the evolution of temporal dynamics.

Introduction

Modern recommender systems, such as those powering Netflix or social media platforms, must process continuous, high-velocity streams of user-item interactions. A primary challenge in this environment is "temporal drift"—the phenomenon where user preferences and item popularities evolve over time. Existing solutions typically fall into two categories: deterministic deep learning models (like RNNs) that capture complex patterns but ignore uncertainty, and probabilistic Bayesian models that handle uncertainty well but are computationally prohibitive for real-time streaming updates.

To address these limitations, we introduce Coupled Variational Recurrent Collaborative Filtering (CVRCF). This framework conjoins the strengths of deep recurrent neural networks and Bayesian probabilistic modeling into a single, scalable architecture capable of on-the-fly updates.

The CVRCF Framework

CVRCF is a dynamic probabilistic factor-based model designed for streaming data. Its architecture is built on several key components:

  1. Latent Factor Decomposition: User and item representations are decomposed into a stationary factor (long-term preferences) and a dynamic factor (short-term tastes).
  2. Temporal Drifting Process: Dynamic factors are modeled as a continuous-time Markov process, where drift is a function of the actual time interval between interactions.
  3. Deep Interaction Network: Rating predictions are modeled as a Gaussian distribution, with parameters generated by deep neural networks fed by the latent factors.
  4. Sequential Variational Inference: To enable efficient Bayesian updates, we propose a Coupled Variational Gated Recurrent Network (CVGRN). This uses coupled GRUs to approximate posterior distributions, avoiding the high cost of traditional particle filtering.
The CVRCF architecture showing (a) the sequential variational inference network and (b) the prediction network.

Experimental Evaluation

We evaluated CVRCF on three benchmark datasets: MovieTweetings (MT), MovieLens-10M (ML-10M), and Netflix. These datasets were segmented chronologically to simulate a streaming environment.

Key Results

CVRCF consistently outperformed state-of-the-art baselines, including both batch models (PMF, time-SVD++) and streaming models (sRec, sRRN).

  • Netflix: CVRCF achieved an RMSE of 0.9050, significantly lower than the best streaming baseline (sRRN at 0.9236).
  • MovieLens-10M: CVRCF reached an RMSE of 0.7831, outperforming sRRN (0.8003).
RMSE performance over time on the Netflix dataset, comparing CVRCF against various baselines.

Insights and Interpretations

The model provides several qualitative advantages:

  • Time-Awareness: By incorporating continuous-time intervals, the model accurately weights the degree of preference drift between interactions.
  • Uncertainty Modeling: The learned variance naturally mirrors real-world phenomena; for instance, uncertainty spikes during "cold-start" periods when new users or items enter the system, then decreases as more data is observed.
  • Evolutionary Dynamics: Visualizing latent factors showed that user preferences drift smoothly, while item popularities tend to change in discrete blocks, validating the model's ability to capture distinct temporal behaviors.

Conclusion

CVRCF represents a significant step in integrating deep learning with probabilistic inference for streaming recommendations. By combining the predictive power of GRUs with the uncertainty handling of Bayesian models, it achieves superior accuracy and robustness. Future work will explore alternative stochastic process assumptions and the integration of Graph Neural Networks to capture more complex relational structures.

Summary updated

Abstract

We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Although deep neural networks have demonstrated the effectiveness of recommendation tasks, it is lack of explorations on integrating probabilistic models and deep architectures under streaming recommendation settings. Conjoining the complementary advantages of probabilistic models and deep neural networks could enhance both model effectiveness and the understanding of inference uncertainties. To bridge the gap, in this paper, we propose a Coupled Variational Recurrent Collaborative Filtering (CVRCF) framework based on the idea of Deep Bayesian Learning to handle the streaming recommendation problem. The framework jointly combines stochastic processes and deep factorization models under a Bayesian paradigm to model the generation and evolution of users' preferences and items' popularities. To ensure efficient optimization and streaming update, we further propose a sequential variational inference algorithm based on a cross variational recurrent neural network structure. Experimental results on three benchmark datasets demonstrate that the proposed framework performs favorably against the state-of-the-art methods in terms of both temporal dependency modeling and predictive accuracy. The learned latent variables also provide visualized interpretations for the evolution of temporal dynamics.
BibTeX
@inproceedings{song2019coupled,
  author = {Qingquan Song and Shiyu Chang and Xia Hu},
  title = {{Coupled Variational Recurrent Collaborative Filtering}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2019},
  url = {https://openreview.net/forum?id=8gOpCnSo-Dz},
}
Figure