Abstract
Multi-task learning (MTL) has been widely applied in computational vision, natural language processing, and other fields, achieving significant performance gains. In recent years, a substantial body of work regarding multi-task learning in recommender systems has emerged, yet a systematic summary of these efforts has been lacking. This survey provides a comprehensive literature review of multi-task recommender systems to help researchers and practitioners navigate the current progress. We introduce the background and motivation, provide a taxonomy based on task relationship discovery, model architecture, and optimization strategy, and discuss applications and promising future directions.
Introduction: The Motivation for Multi-Task Learning
Traditional recommender systems (RS) typically focus on a single objective, such as predicting Click-Through Rate (CTR). However, modern platforms must balance multiple user behaviors, including clicking, purchasing, liking, and dwelling time. Relying on single-task models leads to several critical bottlenecks:
- Data Sparsity: Rare actions like purchases provide insufficient signals for independent model training.
- Sample Selection Bias (SSB): Models trained only on clicked items fail to generalize to the entire space of items shown to users.
- The "Seesaw Phenomenon": Optimizing for one metric (e.g., clicks) often inadvertently degrades another (e.g., conversion rate).
- System Inefficiency: Maintaining separate pipelines for every task is computationally expensive and increases latency during online inference.
Multi-task learning addresses these challenges by allowing models to learn related tasks simultaneously, sharing knowledge across different user behaviors to improve overall robustness and efficiency.
A Taxonomy of Multi-Task Learning in Recommendation
The survey categorizes the MTL recommendation landscape into three primary pillars: task relationship discovery, architecture design, and optimization strategy.
- Task Relationship Discovery: This involves determining what to share between tasks. Approaches range from using prior domain knowledge to automatic discovery through task clustering or gradient-based grouping.
- Architecture Design: This focuses on how to share information. Key paradigms include parameter sharing (hard, soft, and expert-based), transfer learning, and adversarial learning.
- Optimization Strategy: This addresses how to train the model effectively, utilizing techniques like loss weighting (e.g., GradNorm) and gradient surgery (e.g., PCGrad) to prevent conflicting gradients from hindering performance.
Architectural Paradigms for Knowledge Sharing
The choice of architecture is central to the success of MTL. The field has evolved from simple shared layers to complex gating mechanisms designed to mitigate negative interference between tasks.
- Hard Parameter Sharing: Uses shared bottom layers for all tasks, which is efficient but prone to task interference.
- Soft Parameter Sharing: Maintains separate models for tasks but allows "cross-talk" between them to share relevant information.
- Expert Sharing: Models like Multi-gate Mixture-of-Experts (MMoE) and Progressive Layered Extraction (PLE) use gating mechanisms to dynamically route information through specialized "experts."
- Sparse and Contrastive Sharing: These newer methods focus on network pruning and representation robustness to improve scalability and generalization.
Key Insights and Industry Applications
Recent breakthroughs have significantly improved the practicality of MTL in production environments:
- Mitigating the Seesaw Phenomenon: The shift from MMoE to PLE has been pivotal. PLE explicitly separates task-shared experts from task-specific experts, reducing negative transfer where unrelated tasks harm each other's performance.
- Entire Space Modeling: MTL is a primary tool for debiasing. By modeling the transition from Impression → Click → Conversion as a multi-task chain (e.g., ESMM), models can leverage massive "impression" data to improve predictions for rare "conversion" events.
- Benchmarks: The paper highlights the emergence of large-scale benchmarks like Tenrec, which allow for standardized testing of MTL across various recommendation scenarios including e-commerce, social media, and multimedia.
Future Directions and Open Challenges
Despite recent advances, several frontiers remain open for exploration:
- Negative Transfer: Developing more robust methods to automatically prevent tasks from harming each other as the number of tasks scales.
- Multi-task Fusion: Improving how multiple task scores are combined into a final ranked list, potentially using Reinforcement Learning.
- Scalability: Addressing "parameter explosion" through network pruning and more efficient sparse sharing architectures.
- Beyond Accuracy: Integrating fairness, privacy, and explainability as explicit objectives within the MTL framework.
Summary updated