Sequential Recommender via Time-aware Attentive Memory Network

Wendi Ji, Keqiang Wang, Xiaoling Wang, Tingwei Chen, Alexandra I. Cristea2020 · DOI

TL;DR

This paper introduces the Multi-hop Time-aware Attentive Memory (MTAM) network, a novel sequential recommender that explicitly models the temporal dynamics of user behavior. By incorporating time-aware attention and a new recurrent unit (T-GRU), MTAM effectively captures both short-term user intents and long-term preferences, leading to significant improvements in recommendation performance.

What Problem Does This Paper Address?

Recommender systems are crucial for navigating vast item catalogs, but they struggle with two key challenges: understanding the complex temporal nature of user behavior and integrating both immediate (short-term) and enduring (long-term) user preferences. Traditional deep learning models, while powerful, often treat user interactions as mere sequences without adequately accounting for the time intervals between them. This temporal aspect is vital because recent actions are often more indicative of immediate intent than distant ones. Furthermore, users have evolving preferences, making it difficult to balance capturing fleeting interests with remembering stable, long-term tastes.

Methods

The core innovation lies in a temporal gating methodology that enhances both recurrent units and attention mechanisms.

  • Time-aware Recurrent Unit (T-GRU): This new unit modifies the standard GRU by introducing a temporal gate. This gate uses the time interval between consecutive user actions to control how much past information is passed to the current state, effectively filtering out irrelevant historical data and focusing on the immediate short-term intent.
  • Time-aware Attention: This mechanism augments the standard scaled dot-product attention. It incorporates a temporal gate that considers both the time difference and the semantic content of interactions when calculating attention scores. This allows the model to better understand the relevance of past actions based on their temporal proximity.
  • Multi-hop Time-aware Attentive Memory Network (MTAM): This is the overall framework. It uses the T-GRU to encode the user's short-term intent, which then acts as a "query." This query is used to read from a "memory" that stores the user's long-term preferences. The reading process is performed over multiple "hops" using the time-aware attention mechanism. Each hop refines the user representation by attending to different aspects of the long-term memory, allowing for a more nuanced understanding of user preferences.

Datasets

The authors evaluated their model on six real-world datasets:

  • MovieLens-25m: A large-scale rating dataset.
  • Amazon (Electronics, CDs & Vinyl, Movies & TV): Three subsets from the Amazon product review dataset, used for sequential recommendation.
  • Yoochoose: A clickstream dataset from an e-commerce platform.
  • Ali Mobile: A transaction dataset from Alibaba's M-Commerce platform.

These datasets represent both rating-based and transaction-based scenarios, providing a comprehensive evaluation.

Key Insights and Ideas

  • Time is Not Just Order: The paper strongly emphasizes that the time interval between user actions is as crucial as their sequence order for accurate recommendation. Simply processing interactions chronologically isn't enough; understanding how much time has passed is key.
  • Short-term Intent is a Powerful Query: Using the immediate user intent (captured by T-GRU) as a query to search long-term memory is a more effective strategy than previous memory network approaches that used the last item or target item as the query.
  • Multi-hop Memory Reading is Beneficial: Iteratively reading from the user's memory across multiple hops allows the model to capture a richer and more diverse representation of user preferences, especially for longer interaction histories.
  • Temporal Dynamics Matter Across Models: The benefits of time-aware mechanisms aren't limited to one model type. The paper shows that applying time-aware attention to self-attention models (SASRec) also yields significant improvements, particularly on transaction datasets.

Key Results

  • State-of-the-Art Performance: MTAM significantly outperforms several strong baseline models, including traditional RNNs, attention-based models, and existing hybrid recommenders, on five out of six datasets for HR@10 and NDCG@10 metrics.
  • Effectiveness of Components: Ablation studies confirm that both the T-GRU unit and the time-aware attention mechanism contribute substantially to MTAM's performance. The multi-hop structure also generally improves results, though the optimal number of hops varies by dataset.
  • Hybrid Approach Wins: The results highlight that hybrid models combining long-term and short-term preferences, when implemented effectively (like MTAM), are superior to models focusing on only one aspect.

Limitations and Future Work

  • Dataset Dependency for Hops: The optimal number of memory hops is dataset-dependent. For very long sequences (like Ali Mobile), a single hop might be sufficient, suggesting that deeper memory networks aren't universally better.
  • Future Work - Temporal Self-Attention: The authors plan to further explore integrating temporal information more deeply into self-attentive neural recommenders, given the success observed with Time-Aware Self Attention on transaction datasets. They believe this area holds significant potential, especially considering the importance of temporal dynamics in user behavior.

Summary updated

Abstract

Recommendation systems aim to assist users to discover most preferred contents from an ever-growing corpus of items. Although recommenders have been greatly improved by deep learning, they still face several challenges: (1) Behaviors are much more com- plex than words in sentences, so traditional attentive and recurrent models have limitations capturing the temporal dynamics of user preferences. (2) The preferences of users are multiple and evolving, so it is difficult to integrate long-term memory and short-term intent.
BibTeX
@misc{ji2020sequential,
  author = {Wendi Ji and Keqiang Wang and Xiaoling Wang and Tingwei Chen and Alexandra I. Cristea},
  title = {{Sequential Recommender via Time-aware Attentive Memory Network}},
  year = {2020},
  doi = {10.1145/3340531.3411869},
}
Figure