Multi-behavior Self-supervised Learning for Recommendation

Jingcao Xu, Chaokun Wang, Cheng Wu, Yang Song, Kai Zheng, Xiaowei Wang, Changping Wang, Guorui Zhou, Kun Gai

Abstract

Modern recommender systems often deal with a variety of user interactions, such as clicks, forwards, and purchases. To be effective, these engines must fully understand and leverage this multi-behavior data. However, multi-behavior recommendation faces significant challenges: sparse target signals, noisy auxiliary interactions, and an optimization imbalance between self-supervised learning (SSL) tasks and the primary target task. This paper introduces the Multi-Behavior Self-Supervised Learning (MBSSL) framework. MBSSL utilizes a behavior-aware graph neural network with self-attention to capture behavior dependencies, a dual-level SSL paradigm (inter-behavior and intra-behavior) to handle data sparsity and noise, and a customized Hybrid Manipulation on Gradients (HMG) strategy to balance task optimization. Extensive experiments on five real-world datasets show that MBSSL consistently outperforms state-of-the-art baselines.

Introduction

In e-commerce and social media platforms, the "target" behaviors that drive business value—such as purchases or downloads—are often extremely sparse. While users generate a wealth of "auxiliary" data (e.g., clicks, views, or favorites), translating these signals into accurate predictions for the target behavior is difficult. Existing methods often struggle with two core issues:

  1. Noise and Sparsity: Auxiliary data is frequently noisy (a click does not always imply purchase intent), and target data is often too sparse to provide a sufficient training signal.
  2. Optimization Imbalance: When models incorporate auxiliary tasks via self-supervised learning, the gradients from these tasks can conflict with the primary recommendation task. If the model over-optimizes for auxiliary behaviors like clicks, its performance on the target behavior (purchases) may actually degrade.

To address these, MBSSL proposes a robust framework that captures behavioral dependencies while ensuring that auxiliary signals support, rather than distract from, the primary goal.

The MBSSL Framework

1. Behavior-aware Graph Neural Network

MBSSL employs a Graph Neural Network (GNN) enhanced with a self-attention mechanism. Unlike standard GNNs that treat all interactions similarly, this architecture learns the specific dependencies between different behaviors. For instance, it can learn that a "favorite" action is a stronger predictor of a future "purchase" than a simple "page view," allowing the model to weight interactions based on their semantic relevance to the target goal.

2. Dual-Level Self-supervised Learning

To mitigate data sparsity and noise, MBSSL introduces a two-pronged SSL strategy:

  • Inter-behavior SSL: This component contrasts the target behavior with auxiliary behaviors to facilitate knowledge transfer. To ensure high-quality contrastive signals, the framework uses the Swing algorithm to identify and eliminate "false negatives"—users who are behaviorally similar but might be treated as opposites in standard SSL setups.
  • Intra-behavior SSL: This focuses exclusively on the target behavior. By using Edge Dropout (randomly removing connections in the user-item graph) to create augmented views, the model learns to remain robust even when some data points are missing or noisy.

3. Hybrid Manipulation on Gradients (HMG)

The most significant optimization contribution of MBSSL is the HMG strategy. Rather than using static weights for different tasks, HMG adaptively "rectifies" gradients during training. If the gradient from an auxiliary task conflicts with the direction of the target task's gradient, or if its magnitude is disproportionately large, HMG projects the auxiliary gradient onto a normal plane and scales it. This "gradient surgery" ensures that the auxiliary tasks act as a supportive signal without overwhelming the primary recommendation objective.

Experimental Results

Performance Comparison

MBSSL was evaluated against ten state-of-the-art baselines (including LightGCN, GHCF, and CML) across five datasets: Beibei, Taobao, Tmall, IJCAI-Contest, and a short-video dataset from Kuaishou.

  • Significant Gains: On the Beibei and Taobao datasets, MBSSL achieved improvements of 18.64% to 19.71% in Recall@10 and NDCG@10 compared to the strongest baselines.
  • Robustness to Noise: In stress tests where artificial noise was added to auxiliary interactions, MBSSL showed significantly less performance degradation than other models, validating the effectiveness of the intra-behavior SSL and HMG components.
  • Cold-Start Success: The framework proved particularly effective for "inactive" users with very few interactions, a common challenge where traditional collaborative filtering often fails.

Key Insights

  • The Importance of False Negatives: By filtering out false negatives using the Swing algorithm, MBSSL provides a cleaner contrastive signal, preventing the model from trying to push similar users apart in the embedding space.
  • Mitigating Negative Transfer: The combination of inter- and intra-behavior SSL creates a check-and-balance system. This prevents "negative transfer," where learning from auxiliary behaviors like clicks inadvertently harms the model's ability to predict purchases.

Conclusion and Future Work

MBSSL represents a significant step forward in multi-behavior recommendation by addressing the fundamental conflict between auxiliary data volume and target task precision. While the model shows state-of-the-art performance, future work may explore more deeply integrating complex structural information from the user-item graph. Additionally, optimizing the computational overhead of the Swing algorithm and gradient manipulation will be key for deploying MBSSL in massive-scale, real-time production environments.

Summary updated

Abstract

Modern recommender systems often deal with a variety of user interactions, e.g., click, forward, purchase, etc., which requires the underlying recommender engines to fully understand and leverage multi-behavior data from users. Despite recent efforts towards making use of heterogeneous data, multi-behavior recommendation still faces great challenges. Firstly, sparse target signals and noisy auxiliary interactions remain an issue. Secondly, existing methods utilizing self-supervised learning (SSL) to tackle the data sparsity neglect the serious optimization imbalance between the SSL task and the target task. Hence, we propose a Multi-Behavior Self-Supervised Learning (MBSSL) framework together with an adaptive optimization method. Specifically, we devise a behavior-aware graph neural network incorporating the self-attention mechanism to capture behavior multiplicity and dependencies. To increase the robustness to data sparsity under the target behavior and noisy interactions from auxiliary behaviors, we propose a novel self-supervised learning paradigm to conduct node self-discrimination at both inter-behavior and intra-behavior levels. In addition, we develop a customized optimization strategy through hybrid manipulation on gradients to adaptively balance the self-supervised learning task and the main supervised recommendation task. Extensive experiments on five real-world datasets demonstrate the consistent improvements obtained by MBSSL over ten state-of-the art (SOTA) baselines. We release our model implementation at: https://github.com/Scofield666/MBSSL.git.
BibTeX
@inproceedings{xu2023multi,
  author = {Jingcao Xu and Chaokun Wang and Cheng Wu and Yang Song and Kai Zheng and Xiaowei Wang and Changping Wang and Guorui Zhou and Kun Gai},
  title = {{Multi-behavior Self-supervised Learning for Recommendation}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2023},
  doi = {10.1145/3539618.3591734},
}
Figure