Multi-behavior recommendation with SVD Graph Neural Networks

Shengxi Fu, Qianqian Ren, Xingfeng Lv, Jinbao Li

TL;DR

This paper introduces MB-SVD, a novel graph neural network (GNN) approach for multi-behavior recommendation systems. It tackles the challenges of over-smoothing and data sparsity by integrating Singular Value Decomposition (SVD) for enhanced feature representation and a simplified contrastive learning framework.

What Problem Does This Paper Address?

In today's digital landscape, recommendation systems are crucial for filtering vast amounts of information and personalizing user experiences. While single-behavior recommendation models have been effective, they often fall short as user interactions are inherently multi-faceted. For instance, in e-commerce, a user's journey involves not just purchases but also views, add-to-carts, and more. Multi-behavior recommendation aims to leverage these diverse interactions for more accurate predictions. However, existing GNN-based multi-behavior recommendation methods struggle with two key issues:

  1. Over-smoothing: As GNNs propagate information through multiple layers, node embeddings can become too similar, losing their distinctiveness and degrading performance, especially in complex multi-behavior scenarios.
  2. Data Noise and Sparsity: Real-world datasets are often noisy and sparse, making it difficult for models to accurately capture user preferences and generate reliable recommendations.

This paper seeks to address these limitations by proposing a novel framework that effectively integrates multiple behaviors while mitigating the aforementioned challenges.

Methods

The proposed MB-SVD model combines several key techniques:

  • Behavior-Guided Graph Representation: The model first decomposes the multi-behavior graph into individual subgraphs, each representing a specific behavior. Graph Convolutional Networks (GCNs) are then applied to each subgraph to learn behavior-specific user and item embeddings. Residual connections are used to preserve information across layers.
  • Behavior-aware Aggregation: To account for the varying importance of different behaviors, an aggregation module dynamically learns weight scores for each behavior's representation. This allows the model to prioritize more influential behaviors.
  • SVD Augmentation: To further enhance the representations and combat over-smoothing, Singular Value Decomposition (SVD) is employed. The adjacency matrix of each behavior graph is approximated using SVD to capture essential collaborative signals and reconstruct enhanced embeddings. This process helps to distill important features and improve robustness.
  • Simplified Multi-behavior Contrastive Learning: Instead of using multiple augmented views, MB-SVD employs a simplified contrastive learning framework. It directly compares the original weighted multi-behavior embeddings with the SVD-augmented embeddings using the InfoNCE loss. This approach aims to improve representation quality and address data sparsity.
  • Joint Optimization: The model is optimized using a combination of the contrastive loss and a Bayesian Personalized Ranking (BPR) loss, which is a standard for recommendation tasks. This joint optimization aims to leverage the strengths of both approaches for improved performance.

Datasets

The paper evaluates the MB-SVD model on three real-world multi-behavior datasets:

  • Beibei: Encompasses four behaviors: view, favorite, add-to-cart, and purchase.
  • Taobao: Includes four behaviors: click, favorite, add-to-cart, and purchase.
  • Yelp: Features four behaviors: tip, dislike, neutral, and like.

The datasets are split into training, validation, and testing sets. For datasets with more than five discrete behaviors, two are selected for testing/validation, and the rest are used for training.

Key Insights and Ideas

  • SVD as a Regularizer for GNNs: The paper demonstrates that SVD can be effectively used to augment GNN embeddings, helping to mitigate the over-smoothing problem by capturing essential structural information and providing a more robust representation.
  • Simplified Contrastive Learning for Efficiency: By reducing the number of augmented views in contrastive learning, the authors show that it's still possible to achieve strong performance while potentially improving computational efficiency and addressing data sparsity.
  • Behavioral Weighting is Crucial: The behavior-aware aggregation module highlights that not all user behaviors are equally important. Dynamically weighting these behaviors allows the model to better capture user preferences.
  • Synergy of Contrastive and Ranking Losses: The experimental results suggest that combining contrastive learning with a ranking loss (like BPR) is essential for achieving optimal performance in multi-behavior recommendation.

Key Results

The MB-SVD model consistently outperforms several state-of-the-art baselines across the Beibei, Taobao, and Yelp datasets, as measured by Recall@K and NDCG@K metrics. For instance, on the Taobao and Yelp datasets, MB-SVD shows significant improvements over existing methods. The paper also highlights that multi-behavior models generally outperform single-behavior models, underscoring the importance of incorporating diverse interaction types. Ablation studies confirm that removing any behavior or loss component leads to a degradation in performance, emphasizing the effectiveness of the proposed architecture and training strategy.

Limitations and Future Work

The authors acknowledge that while their model shows superior performance, there's always room for improvement. Future work directions suggested include:

  • Learnable Data Augmentation: Investigating techniques for adaptively generating augmented views for the dataset, which could lead to more informative representations.
  • Distillation Techniques: Exploring the use of distillation to incorporate knowledge from external sources, potentially enhancing the robustness and effectiveness of the recommendation model.
  • Investigating Hyperparameter Sensitivity: While the paper explores the impact of key hyperparameters, further in-depth analysis could reveal more nuanced insights.

Summary updated

Abstract

Graph Neural Networks (GNNs) have been extensively employed in the field of recommendation systems, offering users personalized recommendations and yielding remarkable outcomes. Recently, GNNs incorporating contrastive learning have demonstrated promising performance in handling the sparse data problem of recommendation systems. However, existing contrastive learning methods still have limitations in resisting noise interference, especially for multi-behavior recommendation. To mitigate the aforementioned issues, this paper proposes a GNN-based multi-behavior recommendation model called MB-SVD that utilizes Singular Value Decomposition (SVD) graphs to enhance model performance. In particular, MB-SVD considers user preferences across different behaviors, improving recommendation effectiveness. First, MB-SVD integrates the representation of users and items under different behaviors with learnable weight scores, which efficiently considers the influence of different behaviors. Then, MB-SVD generates augmented graph representation with global collaborative relations. Next, we simplify the contrastive learning framework by directly contrasting original representation with the enhanced representation using the InfoNCE loss. Through extensive experimentation, the remarkable performance of our proposed MB-SVD approach in multi-behavior recommendation endeavors across diverse real-world datasets is exhibited.
BibTeX
@inproceedings{fu2024multi,
  author = {Shengxi Fu and Qianqian Ren and Xingfeng Lv and Jinbao Li},
  title = {{Multi-behavior recommendation with SVD Graph Neural Networks}},
  booktitle = {Expert},
  year = {2024},
  doi = {10.1016/j.eswa.2024.123575},
}
Figure