Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks

Elena Smirnova, Flavian Vasile

Introduction

Standard Recurrent Neural Networks (RNNs), such as GRU4Rec, have demonstrated significant success in session-based recommendation by modeling sequences of user interactions. However, these models often treat sequences as simple strings of item IDs, neglecting the rich contextual metadata surrounding each action. A "view" event represents a different user intent than an "add-to-basket" or a "purchase." Similarly, the temporal gap between interactions—whether seconds or days—drastically alters the relevance of historical data.

This paper introduces Contextual RNNs (CRNNs), a framework designed to integrate contextual metadata directly into the network's architecture. By incorporating features like event types and time gaps, CRNNs resolve ambiguities in user intent and significantly improve recommendation performance, particularly in challenging scenarios like post-sale predictions and new item discovery.

Proposed Methodology: Contextual RNNs

The authors propose injecting context into the RNN through two primary mechanisms: context-dependent input/output and context-dependent transition dynamics.

Architecture and Integration

The CRNN architecture modifies the standard recurrent unit to ingest both the current item and its associated context. As shown in the architecture diagram, the model utilizes an Input Module to combine item and context embeddings, a Recurrent Module to maintain state, and an Output Module that leverages the next expected context to refine predictions.

High-level architecture of the Contextual RNN (CRNN) integrating item and context information.

The study explored several integration functions:

  1. Concatenation: Appending context embeddings to item embeddings.
  2. Multiplicative Interaction: Using element-wise multiplication to allow context to "gate" or scale item features.
  3. Context-Dependent Dynamics: Parametrizing the hidden state transitions (e.g., GRU update and reset gates) as functions of the context. This "Context Wrapper" approach allows the model's internal logic to adapt to the situation, such as increasing the "forgetting" rate during long time gaps.

Experimental Evaluation

The models were evaluated on two large-scale e-commerce datasets:

  • YooChoose: A public dataset from the RecSys Challenge 2015, featuring millions of sessions of clicks and purchases.
  • Internal Dataset: A proprietary dataset from Criteo containing three months of activity across diverse verticals, characterized by a significantly larger item catalog (370k items).

Results and Discussion

The primary evaluation metric was Recall@10. The best-performing variant, the Concat-Mult-Context-RNN, achieved a +6.6% lift on YooChoose and a +4.3% lift on the Internal dataset over non-contextual baselines.

Impact of Contextual Features

Context acts as a critical tie-breaker. For instance, knowing a "sale" occurred helps the model transition from suggesting "more of the same" to suggesting complementary items. The CRNN showed a massive +12.5% improvement in predicting items following a sale on the YooChoose dataset.

Uplift in Recall@10 across various contextual features (length, time gap, event type, historical status) for YooChoose and Internal datasets.

Key findings include:

  • Hard Case Performance: CRNNs excel at recommending "new" items (previously unseen by the user), showing a +10.2% improvement in discovery tasks.
  • Dynamics vs. Features: Modifying the recurrent dynamics (how the state evolves) proved more effective than simply adding context as a feature, suggesting that context fundamentally changes how user history is processed.
  • Temporal Sensitivity: The model performs best with very short time gaps (< 2 seconds), indicating that high-frequency context is a strong signal of immediate user intent.

Conclusion and Future Work

CRNNs successfully bridge the gap between context-aware and sequential recommendation. While effective, the authors noted challenges with feature sparsity when using naive context combinations. Future research directions include predicting the next context alongside the item ID—as the next context is currently required for optimal prediction—and exploring longer sequence dependencies within a contextual framework.

Summary updated

BibTeX
@inproceedings{smirnova2017contextual,
  author = {Elena Smirnova and Flavian Vasile},
  title = {{Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks}},
  booktitle = {DLRS@RecSys},
  year = {2017},
  doi = {10.1145/3125486.3125488},
}
Figure