TL;DR
This paper introduces MCGCL, a novel graph contrastive learning framework for multi-label recommendation systems. By combining a holistic user-item graph with homogeneous subgraphs and employing a multi-task learning approach, MCGCL significantly improves recommendation accuracy, especially in challenging scenarios like data sparsity and multi-label classification.
What Problem Does This Paper Address?
Recommending items to users is crucial for businesses, and graph-based structures, particularly bipartite graphs, are effective for modeling user-item interactions. However, traditional methods struggle with data sparsity and the complexity of multi-label classification, where users can have multiple preferences for an item. Existing graph neural network (GNN) approaches face difficulties as data volume increases. This paper tackles these challenges by proposing a novel approach that leverages contrastive learning and a multi-task framework to enhance recommendation effectiveness in multi-label scenarios.
Methods
The core of MCGCL is a two-stage learning process:
- Main Task (Holistic User-Item Graph Learning): This stage focuses on learning representations from the entire user-item bipartite graph. To enrich the learning process and handle data sparsity, two augmented versions of the graph are created through edge removal and random edge addition. These augmented graphs are then processed by separate GCN encoders for different label categories (high, medium, low). Contrastive learning is applied to these augmented graphs to learn representations that capture similarities and differences.
- Subtask (Homogeneous Subgraph Learning): This stage addresses the "hard samples" – those edges that are most difficult to predict. These hard samples are identified using entropy calculated from the main task's predictions. The subtask then focuses on learning representations from homogeneous user-user and item-item subgraphs derived from these hard samples. This helps refine the model's understanding of user preferences and item relationships.
- Combination: The representations learned from both the main and subtasks are combined using an attention aggregation mechanism. This allows the model to effectively leverage information from both the holistic view and the focused view of hard samples.
The framework also incorporates a multi-task learning paradigm, treating the homogeneous subgraph learning as a subtask to the main holistic graph learning.
Datasets
The authors evaluated MCGCL on six Amazon Review datasets: Amazon, Arts, Automotive, Baby, Beauty, and Health. They also included three commonly used datasets for signed bipartite graphs: Review, ML-1M, and Bonanza. For all datasets, nodes with fewer than three connections were filtered out to reduce noise. Ratings were categorized into Low (1-2), Mid (3), and High (4-5). Data was split into 80% for training, 10% for validation, and 10% for prediction.
Key Insights and Ideas
- Contrastive Learning for Multi-Label Recommendations: This paper pioneers the application of contrastive learning specifically for multi-label classification in bipartite graphs, addressing a gap in existing research.
- Multi-Task Learning for Hard Samples: By treating homogeneous subgraph learning as a subtask focused on hard samples, the model can better capture complex relationships and improve prediction accuracy, especially in challenging scenarios.
- Holistic vs. Homogeneous Views: The combination of learning from the entire bipartite graph (holistic) and from specialized homogeneous subgraphs (user-user, item-item) provides a richer representation space.
- Entropy for Hard Sample Identification: Using cross-entropy loss from the main task to identify hard samples is an effective strategy for focusing the subtask's learning efforts.
- Attention Aggregation for Combining Tasks: Attention mechanisms prove to be more effective than simple averaging or MLP-based aggregation for combining representations from the main and subtasks.
Key Results
- Multi-label Classification (Q1): MCGCL significantly outperforms existing end-to-end, contrastive learning, and signed GNN methods across all datasets and metrics (AUC, Macro-F1, Micro-F1). For example, on the Amazon dataset, MCGCL achieved a 12.16% higher AUC than average end-to-end methods.
- Binary Classification (Q2): MCGCL also outperforms SBGCL in binary classification tasks, particularly on larger datasets like Amazon and ML-1M, showing an 8% improvement in AUC.
- Ablation Study (Q3): The ablation study confirms the importance of each component. Removing the main task led to AUC values below 0.5, highlighting its foundational role. Removing the subtask still showed improvements over the "no main task" variant, and removing validation also impacted performance, indicating the synergistic effect of all components.
- Hyperparameter Analysis (Q4): The study shows that MCGCL's performance is sensitive to hyperparameters, but reasonable ranges were identified that enable optimal performance.
- Augmentation Effectiveness: Both edge adding and edge removing were found to be effective augmentation methods. Edge removing generally yielded slightly more stable and higher AUC values.
Limitations and Future Work
- Computational Complexity: While the use of hard samples significantly reduces computational complexity in the subtask, the overall complexity might still be a concern for extremely large-scale graphs.
- Attention Aggregation: The authors suggest that current attention aggregation might not optimally combine the two tasks.
- Future Work: The authors plan to explore task division using a teacher-student training framework for better knowledge exchange and integration between tasks. They also aim to investigate more advanced training skills for large-scale graph data to improve learning efficiency and overall performance.
Summary updated