Coupled Variational Recurrent Collaborative Filtering

Qingquan Song, Shiyu Chang, Xia Hu

TL;DR

Recommender systems need to handle streaming data where user preferences and item popularities constantly evolve. This paper introduces Coupled Variational Recurrent Collaborative Filtering (CVRCF), a framework that merges deep recurrent neural networks with Bayesian probabilistic models to capture both complex temporal dynamics and inference uncertainty. By using a novel sequential variational inference approach, CVRCF efficiently updates in real-time, outperforming state-of-the-art streaming recommendation models.

What Problem Does This Paper Address?

Modern recommender systems (like those on Snapchat, Facebook, or Netflix) don't operate on static datasets; they ingest continuous, high-velocity streams of user-item interactions. This requires models to adapt to "temporal drift"—the reality that a user's tastes change over time, and an item's popularity waxes and wanes.

Currently, researchers tackle this using two main approaches, both of which have flaws:

  1. Deep Learning (e.g., RNNs): Great at capturing complex temporal patterns, but they are usually deterministic. They output point estimates and ignore the inherent uncertainty and randomness of user behavior (like accidental clicks or measurement noise).
  2. Probabilistic (Bayesian) Models: Excellent at handling uncertainty and modeling the data generation process, but notoriously slow and computationally expensive to update in a streaming setting (e.g., using Sequential Monte Carlo methods).

The authors tackle the challenge of conjoining the strengths of deep learning and probabilistic modeling into a single, scalable framework that can be updated on the fly as new data arrives.

Methods

The authors propose CVRCF, a dynamic probabilistic factor-based model. Here is how it works under the hood:

  • Latent Factor Decomposition: User and item representations are split into a stationary factor (capturing long-term, slowly changing preferences) and a dynamic factor (capturing short-term, rapidly evolving tastes).
  • Temporal Drifting Process: The dynamic factors are modeled as a continuous-time Markov process. Crucially, the drift depends on the actual time interval between interactions, not just discrete, uniform time steps.
  • Deep Interaction Network: The actual rating prediction is modeled as a Gaussian distribution. The mean and variance of this distribution are generated by deep neural networks fed by the user and item latent factors.
  • Sequential Variational Inference (The Novelty): To make Bayesian updates computationally feasible for streaming data, they introduce a Coupled Variational Gated Recurrent Network (CVGRN). Instead of using expensive particle filtering, they use coupled GRUs (one chain for users, one for items) to approximate the posterior distributions of the latent factors at each time step. This allows the model to ingest new data and update its beliefs rapidly.

Datasets

The authors evaluated their model on three widely adopted benchmark datasets, segmented chronologically to simulate a streaming environment:

  • MovieTweetings (MT): ~700k ratings extracted from Twitter (2013-2018).
  • MovieLens-10M (ML-10M): 10 million movie ratings (1995-2009).
  • Netflix: The classic challenge dataset with 100 million ratings (1999-2006).

Key Insights and Ideas

  • Time intervals matter: By incorporating the actual continuous-time interval between interactions, the model better captures how much a user's preference might have drifted, rather than assuming all sequential actions have the same temporal weight.
  • Users drift differently than items: Visualizing the learned latent factors revealed that user preferences change smoothly and frequently. In contrast, item popularities change in discrete "blocks" and much more slowly. This validates the architectural choice to separate stationary and dynamic factors.
  • Uncertainty spikes on cold starts: The model's learned variance (uncertainty) naturally decreases as it observes more data over time. However, when an influx of new users or items occurs, the system's uncertainty temporarily spikes before settling down, perfectly mirroring the real-world "cold-start" phenomenon.
  • Deep + Probabilistic = Better: Comparing CVRCF to an ablation model (sRRN, which lacks the probabilistic component) proves that explicitly modeling the generative uncertainty improves predictive accuracy in streaming scenarios.

Key Results

  • CVRCF consistently outperformed five baselines (including batch models like PMF and time-SVD++, and streaming models like sRec and sRRN) across all three datasets.
  • On the massive Netflix dataset, CVRCF achieved an RMSE of 0.9050, compared to the next best streaming baseline (sRRN at 0.9236) and the best batch baseline (time-SVD++ at 0.9311).
  • On MovieLens-10M, CVRCF achieved an RMSE of 0.7831, outperforming sRRN (0.8003).
  • Qualitatively, the model demonstrated highly stable long-term performance. As the streaming timeline progressed, CVRCF maintained the lowest error rate, proving its ability to continuously digest new data without suffering from catastrophic forgetting.

Limitations and Future Work

  • Limitations: To fit the data into the GRU memory, the authors had to truncate user/item sequences into batches, which can slightly limit the RNN's effectiveness. Additionally, the model requires careful tuning of hyperparameters (like training epochs vs. batch iterations) to avoid overfitting the dynamic factors while underfitting the stationary ones.
  • Future Work: The authors plan to explore different stochastic process assumptions for the dynamic factors (moving beyond the current Gaussian-Markov setup). They also suggest incorporating more advanced deep learning architectures, such as Graph Neural Networks (GNNs), to better capture complex user-item relational structures.

Summary updated

Abstract

We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Although deep neural networks have demonstrated the effectiveness of recommendation tasks, it is lack of explorations on integrating probabilistic models and deep architectures under streaming recommendation settings. Conjoining the complementary advantages of probabilistic models and deep neural networks could enhance both model effectiveness and the understanding of inference uncertainties. To bridge the gap, in this paper, we propose a Coupled Variational Recurrent Collaborative Filtering (CVRCF) framework based on the idea of Deep Bayesian Learning to handle the streaming recommendation problem. The framework jointly combines stochastic processes and deep factorization models under a Bayesian paradigm to model the generation and evolution of users' preferences and items' popularities. To ensure efficient optimization and streaming update, we further propose a sequential variational inference algorithm based on a cross variational recurrent neural network structure. Experimental results on three benchmark datasets demonstrate that the proposed framework performs favorably against the state-of-the-art methods in terms of both temporal dependency modeling and predictive accuracy. The learned latent variables also provide visualized interpretations for the evolution of temporal dynamics.
BibTeX
@inproceedings{song2019coupled,
  author = {Qingquan Song and Shiyu Chang and Xia Hu},
  title = {{Coupled Variational Recurrent Collaborative Filtering}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2019},
  url = {https://openreview.net/forum?id=8gOpCnSo-Dz},
}

Loading related content...

Figure