Balancing Gradient Magnitudes in Multi Task Learning

191 concepts138 papers2 proposals

Multi-Task Learning (MTL) promises a highly efficient paradigm for modern machine learning: train a single neural network to handle multiple related objectives simultaneously. As surveyed by Yu Zhang et al. in A Survey on Multi-Task Learning, sharing representations allows models to leverage "hints" across domains, effectively combating data sparsity and reducing overfitting.

Yet, in practice, joint optimization frequently devolves into a destructive tug-of-war. Instead of cooperating, tasks compete for the network's shared capacity. The primary culprit behind this failure is gradient magnitude imbalance (or gradient dominance). When one task produces gradients that are orders of magnitude larger than the others, standard gradient descent updates the shared parameters almost exclusively in favor of that dominant task. The remaining objectives are effectively silenced, leading to a biased backbone and poor overall generalization.

To build an intuitive understanding of why this happens and how to fix it, we must look at the geometry of multi-task updates.


The Geometry of Gradient Dominance

Consider a shared neural network backbone parameterized by $\theta$, which branches into two task-specific heads. The total loss is traditionally a naive sum or average of individual task losses:

$$L_{\text{total}} = L_1 + L_2$$

When we compute the gradient update step, the linearity of differentiation means the total gradient is simply the vector sum of the individual task gradients:

$$G_{\text{total}} = \nabla_{\theta} L_1 + \nabla_{\theta} L_2$$

If Task 1 and Task 2 are balanced, $G_{\text{total}}$ points in a direction that minimizes both losses. However, if $\| \nabla_{\theta} L_1 \| \gg \| \nabla_{\theta} L_2 \|$, the vector sum $G_{\text{total}}$ will align almost perfectly with $\nabla_{\theta} L_1$.

As Liyang Liu et al. observe in Towards Impartial Multi-task Learning, this scale disparity allows high-intensity signals to hijack the optimization process. The network treats the smaller, subtler gradients of the secondary task as negligible noise, causing performance on that task to stagnate or severely degrade.


The Roots of Magnitude Disparity

Why do task gradients vary so wildly in scale? The literature identifies three distinct drivers:

1. Disparate Reward and Loss Scales

In reinforcement learning (RL), different environments naturally yield entirely different reward ranges. Matteo Hessel et al. demonstrate in Multi-Task Deep Reinforcement Learning with PopArt that tasks with large reward magnitudes generate disproportionately massive gradients, drowning out learning signals from tasks with smaller rewards. Aviral Kumar highlights a parallel failure mode in offline Q-learning within Offline Q-learning on Diverse Multi-Task Data Both Scales And Generalizes, noting that standard regression losses become entirely dominated by the tasks with the largest reward scales.

2. Task Difficulty and Learning Curves

Gradients change dynamically as a model learns. Counterintuitively, easier tasks often produce much larger gradient signals early in training. Myungsik Cho shows in Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling that this early imbalance causes the network to overfit to simple objectives, drowning out the subtler signals required to master complex challenges.

Conversely, in image restoration, Ruohan Gao argues in On-demand Learning for Deep Image Restoration that high-corruption instances produce significantly stronger gradients than low-corruption instances, biasing the model toward difficult cases at the expense of fine-grained details.

3. Data Volatility and Outliers

In multivariate time-series forecasting, Liran Nochumsohn notes in A Multi-Task Learning Approach to Linear Multivariate Forecasting that variables with high volatility or large error magnitudes naturally generate steeper gradients, forcing the model to ignore quieter but equally important data streams. A similar phenomenon occurs in object detection, where Jiangmiao Pang et al. show in Libra R-CNN: Towards Balanced Learning for Object Detection that outliers with extreme errors generate gradients that completely overshadow the smaller refinement updates of accurate samples.


Why Magnitude Imbalance Defeats Directional Deconfliction

A significant body of multi-task literature focuses on gradient conflicts—scenarios where two tasks want to push parameters in opposite directions (i.e., a negative cosine similarity). Algorithms like CAGrad, proposed by Bo Liu et al. in Conflict-Averse Gradient Descent for Multi-task learning, seek to resolve these conflicts by projecting gradients onto a shared, non-conflicting direction.

However, recent work reveals that magnitude imbalance is a far more critical hurdle than directional conflict. Zhipeng Zhou et al. argue in Injecting Imbalance Sensitivity for Multi-Task Learning that when task gradients are balanced in magnitude, standard optimization techniques can easily resolve directional conflicts. But if extreme magnitude imbalance exists, the model prioritizes the dominant task regardless of conflict status, leading to Pareto failure where non-dominant tasks stagnate entirely.

Naive Projection with Imbalance:
Task 1 (Dominant):  =========================================>
Task 2 (Sparse):    ↑
Projected Update:   =========================================> (Task 2 is obliterated)

Magnitude-Balanced Projection:
Task 1 (Scaled):    =====>
Task 2 (Scaled):    ↑
Projected Update:   ======↗ (Both tasks contribute equitably)

The geometric reason for this is clear: mathematical projections used to resolve directional interference become severely distorted if vector lengths are highly unequal. As Yang Zhang et al. explain in GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting, adjusting gradient magnitudes before addressing directional conflicts is essential. If magnitudes remain unbalanced, the projection of a tiny vector onto the normal plane of a massive vector can completely obliterate the smaller task's signal.


How Gradient Balancing Restores Equity

To prevent task dominance, modern optimization frameworks employ strategies designed to homogenize or dynamically calibrate updates.

Dynamic Target and Reward Normalization

The PopArt framework (Matteo Hessel et al.) tracks the mean and variance of task targets dynamically, scaling the losses so that their gradients occupy a comparable numerical range. Similarly, Myungsik Cho introduces ARS: Adaptive Reward Scaling for Multi-Task Reinforcement Learning to equalize reward scales dynamically across diverse robotic tasks, preventing high-magnitude rewards from acting as optimization black holes.

Direct Gradient Homogenization

Instead of altering the loss functions, some methods intervene directly at the gradient level. Adrián Javaloy et al. propose RotoGrad in RotoGrad: Gradient Homogenization in Multi-Task Learning, which dynamically re-weights gradients to ensure they maintain similar magnitudes throughout training. This strategy explicitly prioritizes learning from tasks that have converged the least, preventing any single task from locking the shared parameters into a suboptimal state.

Decoupled Optimization Pipelines

Advanced frameworks like GradCraft (Yang Zhang et al.) formalize this by decoupling multi-task optimization into a strict two-step sequence:

  1. Magnitude Calibration: Rescale individual task gradients to establish an equitable playing field.
  2. Directional Deconfliction: Perform global projection and alignment on the scaled vectors.

By ensuring that every task speaks with the same volume before resolving what direction the model should walk, these techniques successfully mitigate gradient dominance—transforming multi-task learning from a destructive tug-of-war into a cooperative optimization process.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_gradient_dominance_gradient_magnitude_im_e6f3fb2a,
  title = {gradient dominance (gradient magnitude imbalance)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-dominance-gradient-magnitude-imbalance/rdr_e6f3fb2a877d490e99d81a6579baae1a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_gradient_dominance_gradient_magnitude_im_e6f3fb2a_v1,
  title = {gradient dominance (gradient magnitude imbalance)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-dominance-gradient-magnitude-imbalance/rdr_e6f3fb2a877d490e99d81a6579baae1a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake