Compressed Interaction Graph based Framework for Multi-behavior Recommendation

Wei Guo, Chang Meng, Enming Yuan, Zhicheng He, Huifeng Guo, Yingxue Zhang, Bo Chen, Yaochen Hu, Ruiming Tang, Xiu Li, Rui Zhang

Abstract

Multi-types of user behavior data (e.g., clicking, adding to cart, and purchasing) are recorded in most real-world recommendation scenarios, which can help to learn users' multi-faceted preferences. However, it is challenging to explore multi-behavior data due to the unbalanced data distribution and sparse target behavior, which lead to the inadequate modeling of high-order relations when treating multi-behavior data "as features" and gradient conflict in multitask learning when treating multi-behavior data "as labels". In this paper, we propose CIGF, a Compressed Interaction Graph based Framework, to overcome the above limitations. Specifically, we design a novel Compressed Interaction Graph Convolution Network (CIGCN) to model instance-level high-order relations explicitly. To alleviate the potential gradient conflict when treating multi-behavior data "as labels", we propose a Multi-Expert with Separate Input (MESI) network with separate input on the top of CIGCN for multi-task learning. Comprehensive experiments on three large-scale real-world datasets demonstrate the superiority of CIGF.

The Challenge of Multi-behavior Data

In modern e-commerce, a "purchase" is a rare event compared to "clicks" or "views." This data sparsity makes it difficult for recommendation engines to learn what a user truly wants. While researchers have tried using these auxiliary behaviors to help, they face two major hurdles:

  1. Inadequate High-Order Modeling: Most models look at behaviors at a "behavior-level" (e.g., "users who click also buy"). They fail to capture "instance-level" nuances—the specific, multi-step paths an individual user takes across different items and actions.
  2. Gradient Conflict: When using Multi-Task Learning (MTL) to predict multiple behaviors simultaneously, the updates for one task (e.g., clicking) often contradict the updates for another (e.g., buying). This "tug-of-war" over the same input features leads to negative transfer, where the model's performance actually degrades.
Distribution of user interactions in the Beibei dataset, illustrating the sparsity of target behaviors like purchases compared to views.

The CIGF Framework

The authors propose the CIGF framework, which consists of two primary innovations designed to solve the issues of high-order modeling and gradient conflict.

CIGCN: Modeling Instance-Level Relations

To model high-order relations (e.g., User A $\rightarrow$ Click $\rightarrow$ Item B $\rightarrow$ Click $\rightarrow$ User C $\rightarrow$ Buy $\rightarrow$ Item D), the model uses matrix multiplication of adjacency matrices. Because this can lead to an exponential explosion of data, they implement a node-wise multi-head attention mechanism. This "compresses" the graph by adaptively selecting only the most relevant high-order paths for each user and item. By modeling the graph at the instance level, the model can identify that a "view" on a specific item is a stronger signal for a specific user than a generic "view" feature.

Illustration of (a) multi-behavior interactions, (b) multi-behavior graph, and (c) the distinction between behavior-level and instance-level relations.

MESI: Decoupling via Separate Inputs

Standard MTL models (like MMOE or PLE) feed the same input into different "experts." CIGF argues this is the root of gradient conflict. Instead, the Multi-Expert with Separate Input (MESI) network uses separate inputs derived from different behavior-specific graphs for each expert. Mathematically, this decouples the gradients, allowing the model to learn from clicks without "confusing" the parameters meant for predicting purchases. This prevents the "seesaw phenomenon" where one task improves at the expense of another.

Experimental Results

The framework was evaluated on three large-scale, real-world e-commerce datasets: Beibei, Taobao, and IJCAI. These datasets are characterized by highly unbalanced distributions where target behaviors are extremely sparse.

  • Superior Accuracy: CIGF consistently outperformed state-of-the-art baselines (like MB-GMN and GHCF). On the Taobao dataset, it achieved a 20.57% improvement in Hit Ratio (HR@10) and a 27.67% improvement in NDCG@10 over the strongest baseline.
  • Efficiency: Despite the complexity of high-order modeling, CIGF is faster than its competitors, showing a 22% to 37% reduction in training time per epoch compared to MB-GMN.
  • Robustness: In tests across user groups with different behavior correlations, CIGF remained stable, whereas other models saw performance drops when behaviors were less correlated, proving its ability to handle "negative transfer."

Conclusion

CIGF introduces a new standard for multi-behavior recommendation by explicitly modeling instance-level high-order relations and effectively mitigating gradient conflict through input separation. While improvements vary based on behavior overlap in datasets, the framework's ability to handle sparse target behaviors and complex interaction paths makes it a robust solution for modern recommendation systems.

Summary updated

Abstract

Multi-types of user behavior data (e.g., clicking, adding to cart, and purchasing) are recorded in most real-world recommendation scenarios, which can help to learn users' multi-faceted preferences. However, it is challenging to explore multi-behavior data due to the unbalanced data distribution and sparse target behavior, which lead to the inadequate modeling of high-order relations when treating multi-behavior data ''as features'' and gradient conflict in multitask learning when treating multi-behavior data ''as labels''. In this paper, we propose CIGF, a Compressed Interaction Graph based Framework, to overcome the above limitations. Specifically, we design a novel Compressed Interaction Graph Convolution Network (CIGCN) to model instance-level high-order relations explicitly. To alleviate the potential gradient conflict when treating multi-behavior data ''as labels'', we propose a Multi-Expert with Separate Input (MESI) network with separate input on the top of CIGCN for multi-task learning. Comprehensive experiments on three large-scale real-world datasets demonstrate the superiority of CIGF. Ablation studies and in-depth analysis further validate the effectiveness of our proposed model in capturing high-order relations and alleviating gradient conflict. The source code and datasets are available at https://github.com/MC-CV/CIGF.
BibTeX
@inproceedings{guo2023compressed,
  author = {Wei Guo and Chang Meng and Enming Yuan and Zhicheng He and Huifeng Guo and Yingxue Zhang and Bo Chen and Yaochen Hu and Ruiming Tang and Xiu Li and Rui Zhang},
  title = {{Compressed Interaction Graph based Framework for Multi-behavior Recommendation}},
  booktitle = {The Web Conference},
  year = {2023},
  doi = {10.1145/3543507.3583312},
}
Figure