Personalized Graph Neural Networks With Attention Mechanism for Session-Aware Recommendation

Mengqi Zhang, Shu Wu, Meng Gao, Xin Jiang, Ke Xu, Liang WangIEEE TKDE 2022 · OpenReview

TL;DR

This paper introduces A-PGNN, a novel recommendation system that leverages personalized graph neural networks and an attention mechanism to better predict user behavior in sessions. By explicitly modeling both short-term (current session) and long-term (historical sessions) user preferences, A-PGNN significantly outperforms existing methods on real-world datasets.

What Problem Does This Paper Address?

Recommender systems are crucial for navigating the vast amount of online information. Session-aware recommendation aims to predict a user's next action within a current browsing session, considering both their immediate interests and their historical behavior. Existing methods often struggle to capture the complex relationships between items within a session and, more importantly, to effectively distinguish the varying influence of different historical sessions on current user intent. This can lead to suboptimal recommendations, especially when user interests evolve or when historical context is highly relevant but not properly weighted.

Methods

The proposed A-PGNN model has two core components:

  • Personalized Graph Neural Network (PGNN): Unlike traditional Graph Neural Networks (GNNs) that treat all users similarly, PGNN constructs a unique "user behavior graph" for each user. This graph captures item transition relationships within and across a user's historical and current sessions. Crucially, PGNN injects user-specific embeddings into the graph learning process, allowing it to learn personalized item transition patterns. This is achieved by fusing user embeddings with node embeddings during the graph propagation steps.
  • Dot-Product Attention Mechanism: This component, inspired by the Transformer architecture, explicitly models the impact of historical sessions on the current session. It uses the current session's item embeddings as queries to attend to historical session embeddings, effectively learning how much weight to give to different historical interactions when predicting the next item. This allows the model to dynamically adjust the influence of past behavior based on its relevance to the current context.

The model then combines these learned representations with a user's static embedding to generate a unified user representation for final recommendation.

Datasets

The authors evaluated A-PGNN on two real-world datasets:

  • Xing: This dataset, sourced from RecSys Challenge 2016, contains job-posting interactions (click, bookmark, reply, delete) over an 80-day period. It was preprocessed to define sessions based on a 30-minute idle threshold and to filter out noisy or short interactions.
  • Reddit: This dataset consists of user interactions (comments) on subreddits, extracted from a social news and discussion website. Sessions were defined using a 60-minute idle threshold.

Key Insights and Ideas

  • Personalization is Key: Explicitly modeling user-specific item transition patterns through PGNN significantly improves recommendation performance compared to generic graph-based approaches.
  • Contextual Weighting of History: The attention mechanism is vital for dynamically weighing the influence of historical sessions. Not all past interactions are equally relevant to the current session, and A-PGNN's ability to discern this is a major advantage.
  • "More is Not Always Better": The study on the number of historical sessions revealed that while incorporating more history can be beneficial, there's a point of diminishing returns. For certain datasets (like Reddit), too much historical data can introduce noise and degrade performance, highlighting the importance of the attention mechanism in filtering relevant information.
  • Session Length Matters, but A-PGNN is Robust: A-PGNN demonstrates stable performance across varying current session lengths, outperforming other models, especially for longer sessions where capturing evolving interests is more critical.

Key Results

A-PGNN consistently outperformed state-of-the-art session-aware recommendation methods, including SR-GNN, H-RNN, and HierTCN, across both datasets.

  • On the Xing dataset, A-PGNN achieved the best results in terms of Recall@5, 10, 20 and Mrr@5, 10, 20. For instance, it achieved a Recall@5 of 14.38% and Mrr@5 of 10.36%.
  • On the Reddit dataset, A-PGNN also demonstrated superior performance, with Recall@5 of 49.19% and Mrr@5 of 33.54%.
  • Ablation studies confirmed the effectiveness of both the PGNN component (A-PGNN(-A)) and the Dot-Product Attention mechanism (A-PGNN(-P)), with significant performance drops when either was removed. The combined model (A-PGNN) was superior to any single component or the baseline session-based model (A-PGNN(-A-P)).

Limitations and Future Work

  • Limitations: The paper notes that while A-PGNN is effective, the "over-smoothing" phenomenon in graph neural networks can be a concern with very deep propagation steps, particularly on datasets like Xing where item categories are more concentrated.
  • Future Work: The authors suggest improving the flexibility and scalability of PGNN by incorporating dynamic graph neural networks. They also express interest in exploring more effective attention mechanisms to better integrate users' long- and short-term interests.

Summary updated

Abstract

The problem of session-aware recommendation aims to predict users’ next click based on their current session and historical sessions. Existing session-aware recommendation methods have defects in capturing complex item transition relationships. Other than that, most of them fail to explicitly distinguish the effects of different historical sessions on the current session. To this end, we propose a novel method, named Personalized Graph Neural Networks with Attention Mechanism (A-PGNN) for brevity. A-PGNN mainly consists of two components: one is Personalized Graph Neural Network (PGNN), which is used to extract the personalized structural information in each user behavior graph, compared with the traditional Graph Neural Network (GNN) model, which considers the role of the user when the node embedding is updated. The other is Dot-Product Attention mechanism, which draws on the Transformer net to explicitly model the effect of historical sessions on the current session. Extensive experiments conducted on two real-world data sets show that A-PGNN evidently outperforms the state-of-the-art personalized session-aware recommendation methods.
BibTeX
@article{zhang2022personalized,
  author = {Mengqi Zhang and Shu Wu and Meng Gao and Xin Jiang and Ke Xu and Liang Wang},
  title = {{Personalized Graph Neural Networks With Attention Mechanism for Session-Aware Recommendation}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2022},
  url = {https://openreview.net/forum?id=f0JgPpMK7L},
}
Figure