Compressed Interaction Graph based Framework for Multi-behavior Recommendation

Wei Guo, Chang Meng, Enming Yuan, Zhicheng He, Huifeng Guo, Yingxue Zhang, Bo Chen, Yaochen Hu, Ruiming Tang, Xiu Li, Rui Zhang

TL;DR

CIGF is a novel recommendation framework that leverages "auxiliary" user actions (like clicking or adding to cart) to better predict "target" actions (like purchasing). It introduces a compressed graph convolution method to capture complex, high-order user-item relationships and a multi-task learning architecture that prevents different behaviors from interfering with each other's learning process.

What Problem Does This Paper Address?

In modern e-commerce, a "purchase" is a rare event compared to "clicks" or "views." This data sparsity makes it difficult for recommendation engines to learn what a user truly wants. While researchers have tried using these auxiliary behaviors to help, they face two major hurdles:

  1. Inadequate High-Order Modeling: Most models look at behaviors at a "behavior-level" (e.g., "users who click also buy"). They fail to capture "instance-level" nuances—the specific, multi-step paths an individual user takes across different items and actions.
  2. Gradient Conflict: When using Multi-Task Learning (MTL) to predict multiple behaviors simultaneously, the updates for one task (e.g., clicking) often contradict the updates for another (e.g., buying). This "tug-of-war" over the same input features leads to negative transfer, where the model's performance actually degrades.

Methods

The authors propose the CIGF framework, which consists of two primary innovations:

  • CIGCN (Compressed Interaction Graph Convolution Network): To model high-order relations (e.g., User A $\rightarrow$ Click $\rightarrow$ Item B $\rightarrow$ Click $\rightarrow$ User C $\rightarrow$ Buy $\rightarrow$ Item D), the model uses matrix multiplication of adjacency matrices. Because this can lead to an exponential explosion of data, they implement a node-wise multi-head attention mechanism. This "compresses" the graph by adaptively selecting only the most relevant high-order paths for each user and item.
  • MESI (Multi-Expert with Separate Input): Standard MTL models (like MMOE or PLE) feed the same input into different "experts." CIGF argues this is the root of gradient conflict. Instead, MESI uses separate inputs derived from different behavior-specific graphs for each expert. Mathematically, this decouples the gradients, allowing the model to learn from clicks without "confusing" the parameters meant for predicting purchases.

Datasets

The framework was evaluated on three large-scale, real-world e-commerce datasets:

  • Beibei: Includes View, Cart, and Buy behaviors.
  • Taobao: Includes View, Favorite, Cart, and Buy behaviors.
  • IJCAI: Includes View, Favorite, Cart, and Buy behaviors.

These datasets are characterized by highly unbalanced distributions (e.g., in Taobao, there are roughly 7.6 million interactions for 147k users, but "Buy" actions are extremely sparse).

Key Insights and Ideas

  • Instance-Level vs. Behavior-Level: The "aha moment" here is that the specific sequence of interactions matters more than the general category of behavior. By modeling the graph at the instance level, the model can identify that a "view" on a specific item is a stronger signal for a specific user than a generic "view" feature.
  • Decoupling via Separate Inputs: The most significant theoretical contribution is the proof that using separate inputs for different tasks in an MTL setting effectively eliminates the "coupled gradient" issue. This allows for more stable training and prevents the "seesaw phenomenon" where one task improves at the expense of another.
  • Graph Compression as Feature Selection: High-order graphs are computationally expensive. Using attention to compress these graphs acts as an automated feature selection process, identifying which multi-hop relationships (like "friends of friends who bought this") are actually predictive.

Key Results

  • Superior Accuracy: CIGF consistently outperformed state-of-the-art baselines (like MB-GMN and GHCF). On the Taobao dataset, it achieved a 20.57% improvement in Hit Ratio (HR@10) and a 27.67% improvement in NDCG@10 over the strongest baseline.
  • Efficiency: Despite the complexity of high-order modeling, CIGF is faster than its competitors. It showed a 22% to 37% reduction in training time per epoch compared to MB-GMN.
  • Robustness to Correlation: In tests across user groups with different behavior correlations, CIGF remained stable, whereas other models saw performance drops when behaviors were less correlated, proving its ability to handle "negative transfer."

Limitations and Future Work

  • Dataset Overlap: The authors noted that improvements on the Beibei dataset were less dramatic (1.30% HR improvement). This is because Beibei has a very high overlap between behaviors (users almost always view and cart an item before buying), reducing the need for complex high-order modeling.
  • Future Directions: The paper suggests exploring more sophisticated graph convolution operators within the CIGF framework and investigating how to handle even more diverse types of user feedback (like "dislikes" or "returns") which might introduce even stronger gradient conflicts.

Summary updated

Abstract

Multi-types of user behavior data (e.g., clicking, adding to cart, and purchasing) are recorded in most real-world recommendation scenarios, which can help to learn users' multi-faceted preferences. However, it is challenging to explore multi-behavior data due to the unbalanced data distribution and sparse target behavior, which lead to the inadequate modeling of high-order relations when treating multi-behavior data ''as features'' and gradient conflict in multitask learning when treating multi-behavior data ''as labels''. In this paper, we propose CIGF, a Compressed Interaction Graph based Framework, to overcome the above limitations. Specifically, we design a novel Compressed Interaction Graph Convolution Network (CIGCN) to model instance-level high-order relations explicitly. To alleviate the potential gradient conflict when treating multi-behavior data ''as labels'', we propose a Multi-Expert with Separate Input (MESI) network with separate input on the top of CIGCN for multi-task learning. Comprehensive experiments on three large-scale real-world datasets demonstrate the superiority of CIGF. Ablation studies and in-depth analysis further validate the effectiveness of our proposed model in capturing high-order relations and alleviating gradient conflict. The source code and datasets are available at https://github.com/MC-CV/CIGF.
BibTeX
@inproceedings{guo2023compressed,
  author = {Wei Guo and Chang Meng and Enming Yuan and Zhicheng He and Huifeng Guo and Yingxue Zhang and Bo Chen and Yaochen Hu and Ruiming Tang and Xiu Li and Rui Zhang},
  title = {{Compressed Interaction Graph based Framework for Multi-behavior Recommendation}},
  booktitle = {The Web Conference},
  year = {2023},
  doi = {10.1145/3543507.3583312},
}

Loading related content...

Figure