Recurrent Latent Variable Networks for Session-Based Recommendation

Sotirios P. Chatzis, Panayiotis Christodoulou, Andreas S. AndreouDLRS@RecSys 2017 · OpenReview

Abstract

This paper introduces ReLaVaR, a novel framework that transforms deterministic Recurrent Neural Networks (RNNs) into stochastic latent variable models for session-based recommendation. By employing Amortized Variational Inference to account for uncertainty, ReLaVaR effectively addresses the challenge of data sparsity in short user sessions. Experimental results on the RecSys Challenge 2015 benchmark demonstrate that ReLaVaR significantly outperforms existing state-of-the-art methods in ranking accuracy (MRR@20), while maintaining computational efficiency suitable for real-time systems.

Introduction

Session-based recommendation is a critical task in digital services where user profiles are often unavailable or anonymous. Unlike traditional recommendation systems that rely on long-term user history, session-based models must predict the next item based solely on a short sequence of current interactions.

The primary challenge in this domain is data sparsity. When a session contains only a few clicks, the signal is weak and the uncertainty regarding user intent is high. While standard Recurrent Neural Networks (RNNs), such as Gated Recurrent Units (GRUs), excel at modeling sequential dependencies, they are inherently deterministic. This makes them prone to overfitting on noisy, short-term data. To address this, the authors propose ReLaVaR (Recurrent Latent Variable Network for Recommendation), a framework that integrates stochastic latent variables into the RNN architecture to better account for uncertainty.

Methodology

ReLaVaR transforms the deterministic hidden states of a GRU into stochastic variables. Instead of representing a session state as a single point in vector space, the model maintains a probability distribution over possible states.

Stochastic Hidden States and Variational Inference

In ReLaVaR, the hidden units are treated as latent variables. To perform inference efficiently at scale, the authors utilize Amortized Variational Inference (AVI). A "recognition" network—also implemented as a GRU—learns to map the input sequence to the parameters (mean and variance) of the posterior distribution. During training, the reparameterization trick is used to ensure the model remains end-to-end differentiable, allowing gradients to flow through the stochastic sampling process.

Ranking and Loss Functions

While pairwise loss functions are common in recommendation, the authors found that a pointwise approach using a Multinoulli distribution (Cross-Entropy loss) yielded superior results when combined with the stochastic architecture. This suggests that the model's ability to represent uncertainty complements the standard classification objective.

Impact of Latent Units on ReLaVaR performance (Recall@20 and MRR@20) using Cross-Entropy loss.

Experimental Evaluation

The model was evaluated on the RecSys Challenge 2015 (YOOCHOOSE) dataset, a large-scale benchmark containing over 31 million click actions. The dataset is characterized by high sparsity, with nearly 37,500 unique items.

Key Results

ReLaVaR was compared against several state-of-the-art baselines, including BPR-MF and advanced GRU4Rec variants (M2, M4).

Comparison of ReLaVaR against baseline models on the YOOCHOOSE dataset.

As shown in the results table, ReLaVaR achieved a significant improvement in ranking accuracy, reaching an MRR@20 of 0.3527. This outperforms the standard GRU (0.2693) and the previous best-performing M2 variant (0.3091). While the Recall@20 of 0.6507 was a substantial gain over the baseline GRU, it was slightly lower than the M2 variant. This indicates that ReLaVaR is particularly effective at ranking the correct item higher in the list, even if its overall retrieval coverage is slightly different from some deterministic variants.

Discussion and Insights

A key finding of this work is that "uncertainty as a feature" helps prevent overfitting in sparse sessions. By treating hidden states as distributions, the model effectively regularizes itself against the noise of short click sequences.

Interestingly, the authors observed that stacking multiple GRU layers did not improve performance, likely because the temporal dynamics of short sessions do not require complex multi-scale modeling. Furthermore, despite the overhead of sampling, ReLaVaR remains computationally efficient, with an average prediction time of 0.844ms per click, making it suitable for real-time production environments.

Conclusion

ReLaVaR demonstrates that bridging deep sequential modeling with Bayesian uncertainty can yield significant gains in session-based recommendation. Future work may explore the application of this stochastic approach to longer sequences or its effectiveness in addressing the "cold-start" problem for new items.

Summary updated

BibTeX
@inproceedings{chatzis2017recurrent,
  author = {Sotirios P. Chatzis and Panayiotis Christodoulou and Andreas S. Andreou},
  title = {{Recurrent Latent Variable Networks for Session-Based Recommendation}},
  booktitle = {DLRS@RecSys},
  year = {2017},
  url = {https://openreview.net/forum?id=-Y5A2x1lpF},
}
Figure