Introduction
In modern recommendation systems, models are rarely tasked with a single objective. Instead, they must simultaneously predict multiple user behaviors, such as click-through rate (CTR), dwell time, and engagement metrics like likes or shares. This multi-task learning (MTL) approach is powerful but faces two significant hurdles: the seesaw phenomenon and negative transfer. The seesaw phenomenon occurs when optimizing for one task inadvertently degrades the performance of another due to shared underlying neural layers. Negative transfer happens when noisy or irrelevant information from a secondary task pollutes the learning process of the primary task.
DRGrad (Direct Routing Gradient) is a novel MTL framework designed to perform "information surgery" to resolve these conflicts. By dynamically routing helpful gradients between tasks and employing personalized gating, DRGrad achieves significant performance gains in massive, real-world recommendation systems without increasing serving latency.
The DRGrad Framework
The DRGrad framework introduces three specialized components that work together during the training process to manage gradient flow and task interference.
Core Components
- Split-MMoE Architecture: Moving beyond standard shared-bottom approaches, DRGrad splits the primary task into a dedicated layer (exclusive to that task) and a shared layer (shared with auxiliary tasks). This structure provides a baseline for isolating task-specific signals.
- The Router Network: Acting as the "surgeon," this module calculates the cosine similarity between gradients of different tasks during backpropagation. If gradients are "cooperative" (pointing in similar directions), the router allows information flow. If they conflict, it blocks or modifies the gradient to prevent interference.
- The Updater Network: This module acts as a dynamic balancer. It tracks accumulated gradient signals and uses a softmax-based mechanism to adjust the weights of dedicated versus shared components in real-time.
- Personalized Gate Network: Inspired by PPNet, this component injects user-specific features (e.g., User ID, Item ID) directly into the shared layers. This allows the model to resolve gradient conflicts differently for different users, recognizing that what constitutes "noise" for one user might be a valuable "signal" for another.
Gradient Routing Logic
At the heart of DRGrad is the insight that gradients serve as a compass for model updates. Rather than just weighting final losses, DRGrad examines the direction of updates at every step of training.
As shown in the logic above, when the angle $\theta$ between gradients $g_1$ and $g_2$ is greater than $90^o$, the tasks are in conflict. When $\theta < 90^o$, they are cooperative. By adding personalized gating, the model can perform this "surgery" with a finer scalpel, acknowledging that task conflicts are often user-dependent.
Experimental Validation
The authors validated DRGrad across three distinct environments to ensure robustness and scalability:
- Industrial Dataset: A massive dataset of 15 billion samples from a live application.
- UCI Census-Income Dataset: A standard public benchmark for predicting income and marital status.
- Synthetic Dataset: A custom environment designed to simulate absolute conflict and cooperation to verify the Router's logic.
Key Results
DRGrad demonstrated high effectiveness at scale. On the 15-billion-sample industrial dataset, it achieved a +0.25% gain in Click AUC and a +0.12% gain in Dwell Time AUC. In live A/B testing, the model delivered a 1.79% increase in CTR and a 0.57% increase in Dwell Time, significantly outperforming established models like MMoE and PLE. Crucially, DRGrad is designed to perform its complex logic during training, leaving inference speed virtually unchanged.
Discussion and Future Work
While DRGrad effectively mitigates the seesaw effect, some areas for exploration remain. The current "Split-MMoE" structure requires manual partitioning of tasks, which could potentially be automated via neural architecture search (NAS). Additionally, while the framework is optimized for scenarios with a clear primary task, adapting it to environments with multiple equally important tasks may require more complex routing logic. Finally, extremely high-noise auxiliary tasks still pose a challenge that the router must adapt to quickly to protect shared layers.
Summary updated