Gradient Surgery for Multi-Task Learning

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, Chelsea Finn

Introduction

Multi-task learning (MTL) aims to train a single model to perform multiple tasks simultaneously, leveraging shared features to improve generalization and data efficiency. However, in practice, MTL often underperforms compared to training separate models for each task. This performance gap is frequently attributed to gradient interference, where gradients from different tasks conflict during the optimization process. This paper introduces PCGrad (Projecting Conflicting Gradients), a "gradient surgery" technique designed to mitigate these conflicts by projecting gradients onto each other's normal planes.

The "Tragic Triad" of MTL Optimization

The authors identify three specific conditions that create a destructive optimization environment in multi-task settings, termed the "Tragic Triad":

  1. Conflicting Gradients: Task gradients point in opposite directions (negative cosine similarity).
  2. High Curvature: The loss landscape contains sharp, narrow valleys.
  3. Magnitude Imbalance: One task's gradient is significantly larger than others, allowing it to dominate the update and "bully" smaller tasks.

When these conditions coincide, standard optimizers like Adam or SGD overestimate progress on the dominant task while inadvertently damaging performance on others, leading to stalled learning.

PCGrad: Gradient Surgery

The proposed solution, PCGrad, is a model-agnostic algorithm that modifies gradients before they are passed to the optimizer. The core intuition is to eliminate the component of a task's gradient that directly opposes another task's gradient.

The PCGrad Algorithm: 1. Compute individual gradients $g_i$ and $g_j$ for tasks $i$ and $j$. 2. Determine if a conflict exists by checking if their dot product is negative ($g_i \cdot g_j < 0$). 3. If they conflict, project $g_i$ onto the normal plane of $g_j$: $$g_i = g_i - \frac{g_i \cdot g_j}{\|g_j\|^2} g_j$$ 4. This process is repeated for all task pairs in a randomized order to ensure an unbiased update.

The PCGrad surgery: projecting conflicting gradient $g_i$ onto the normal plane of $g_j$ to create a non-conflicting update.

Optimization Landscape Analysis

To visualize the impact of PCGrad, the authors analyzed a toy multi-task objective. In standard optimization, the path often gets stuck or oscillates in high-curvature regions where task gradients conflict. PCGrad allows the optimizer to navigate these "valleys" more effectively by ensuring that the update direction does not actively harm any individual task's objective.

Visualization of the multi-task objective landscape (a-c) and optimization paths for Adam (d) vs. Adam + PCGrad (e). PCGrad successfully navigates the conflicting region to reach the global optimum.

Experimental Results

The effectiveness of PCGrad was demonstrated across several domains:

Reinforcement Learning

In the Meta-World MT50 benchmark, which involves 50 distinct robotic manipulation tasks, a standard Soft Actor-Critic (SAC) agent solved fewer than 40% of tasks. Integrating PCGrad increased the success rate to ~70%, outperforming even independent agents trained on single tasks. This suggests that PCGrad is crucial in RL, where gradient noise from complex tasks can otherwise prevent the learning of simpler ones.

Supervised Learning

  • NYUv2 & CityScapes: When combined with state-of-the-art architectures like MTAN, PCGrad achieved top results in multi-task scene understanding (segmentation, depth, and surface normals).
  • CIFAR-100: PCGrad improved test accuracy from 67.7% to 77.5% when used with routing networks, acting as a powerful regularizer that forces the model to find universally useful representations.

Conclusion

PCGrad offers a simple yet effective way to handle the inherent tensions in multi-task optimization. By performing "surgery" on conflicting gradients, it prevents destructive interference and enables models to find better minima. While it introduces some computational overhead due to per-task gradient computation, the gains in generalization and stability—particularly in complex RL environments—make it a vital tool for modern MTL research.

Summary updated

BibTeX
@inproceedings{yu2020gradient,
  author = {Tianhe Yu and Saurabh Kumar and Abhishek Gupta and Sergey Levine and Karol Hausman and Chelsea Finn},
  title = {{Gradient Surgery for Multi-Task Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=020HornOxJ0},
}
Figure