Noise-Aware MMSE Combining of Multi-Task Gradients

When one neural network is trained on several tasks at once, the per-task gradients often point in conflicting directions. Popular fixes either delete the conflicting component outright or limit how far the update may stray from the plain average. Wireless receivers faced the same problem decades ago. A multi-antenna receiver must pull one signal out of a mixture of other signals and noise. Fully cancelling the interferers ("zero-forcing") is known to amplify noise. The minimum mean-square error (MMSE) receiver, a linear filter that minimizes expected squared error, instead weighs cancellation against noise, and in doing so avoids that amplification. We propose to treat the per-task gradients as interfering streams and the mini-batch sampling noise as receiver noise. The update direction is then the MMSE solution. It approaches joint conflict cancellation when gradients are precise, approaches plain averaging when they are noisy, and gets its trade-off from a measured noise level rather than a hand-tuned constant.

The bet is narrow and testable. If gradient noise is what makes hard projection harmful, the new method's advantage over existing conflict-resolution methods should grow as batches shrink and gradients get noisier, and it should disappear at large batch sizes. If it does not, the wireless analogy is decorative, and we will say so.

Core Claim

Let $\hat G = [\hat g_1, \dots, \hat g_K]$ be the mini-batch gradients of the $K$ task losses with respect to the shared parameters. Let $\Sigma = \sum_k \Sigma_k$ be the summed covariance of their sampling noise, and let $t$ be a target vector of desired first-order improvements per task (all ones, meaning "equal progress on every task"). We choose the update $d$ to minimize the expected squared gap between the improvements it actually delivers, $G^\top d$ with $G$ the true gradients, and the target. Under a zero-mean noise model this expected loss equals $\|\hat G^\top d - t\|^2 + d^\top \Sigma d$, and its minimizer is

$$d^\star = (\hat G \hat G^\top + \Sigma)^{-1} \hat G\, t = \Sigma^{-1} \hat G \left(I_K + \hat G^\top \Sigma^{-1} \hat G\right)^{-1} t .$$

The second form, from the matrix inversion identity, needs only a $K \times K$ inverse when $\Sigma$ is diagonal, so cost does not depend on the parameter count. This is the MMSE receiver formula $(\mathbf{H}\mathbf{H}^H + \sigma^2 \mathbf{I})^{-1}\mathbf{h}$ with task gradients in the role of channels.

The two limits match the wireless story exactly. As $\Sigma \to 0$, $d^\star = \hat G(\hat G^\top \hat G)^{-1} t$, the joint zero-forcing direction that gives every task exactly equal first-order improvement. Its norm blows up when two task gradients are nearly collinear but opposed, which is the gradient version of noise enhancement. When the noise dominates, $d^\star \propto \Sigma^{-1}\hat G t$, a noise-preconditioned sum of task gradients, the analogue of maximum ratio combining.

Two further properties come for free. Because $d^\star$ is linear in $t$, rescaling $t$ does not shift the trade-off, so the method has no conflict-strength hyperparameter. And because $\Sigma$ sums per-task noise, a task with high-variance gradients, such as a sparse-reward RL task, is automatically trusted less.

Relation to Prior Work

The wireless literature supplies the mechanism and its motivating evidence. Outage Probability of Dual-Hop Multiple Antenna AF Systems with Linear Processing in the Presence of Co-Channel Interference proves that zero-forcing loses diversity order (from $N$ to $N-M$ with $M$ interferers) while MMSE keeps the full $N$. URLLC With Massive MIMO: Analysis and Design at Finite Blocklength shows the other extreme also fails: plain maximum-ratio aggregation cannot reach even 50% network availability, where MMSE exceeds 90%. These results motivate the proposal but do not transfer automatically. In gradient space the parameter dimension vastly exceeds $K$, so "running out of antennas" is not the failure mode. The relevant pathology is instead the poor conditioning of $\hat G^\top \hat G$ when two task gradients nearly oppose each other.

The multi-task literature supplies the competitors and a key constraint. Gradient Surgery for Multi-Task Learning (PCGrad) removes, pairwise and sequentially, the component of each task gradient that conflicts with another. It behaves like a hard, noise-blind zero-forcer, though not identical to our joint $\Sigma \to 0$ limit. Conflict-Averse Gradient Descent for Multi-task learning (CAGrad) maximizes the worst task's improvement within a ball of fixed radius $c$ around the average gradient. It shows that pure Pareto-seeking updates can land at points with poor average loss, a negative result that constrains our design: our update must not abandon average descent. CAGrad also already interpolates between averaging and conflict avoidance, so it is the main threat. The difference is that its interpolation constant is static and tuned, while ours is set at every step by measured noise relative to measured conflict. RotoGrad: Gradient Homogenization in Multi-Task Learning aligns gradients by rotating shared features and balancing magnitudes, but must truncate rotations to about 1024 dimensions. Our $K \times K$ solve sidesteps that cost.

Existing bridges between the fields run the other way. Learning the MMSE Channel Estimator, Learning The MMSE Channel Predictor, and Lower Bounds for the Minimum Mean-Square Error via Neural Network-based Estimation use neural networks to approximate or bound MMSE estimators. None of them uses MMSE as an optimizer rule for learning. A Robust Diffusion Minimum Kernel Risk-Sensitive Loss Algorithm over Multitask Sensor Networks applies mean-square criteria to distributed multi-task filtering over sensor nodes, not to combining gradients in a shared network.

Research Program

We will build the combiner as a drop-in replacement for PCGrad and CAGrad, operating on the shared-parameter gradients they already compute. That requires $K$ backward passes per step, the same as theirs.

The open engineering question is estimating $\Sigma$ cheaply. Our primary estimator is diagonal and comes from per-task running first and second moments, the same quantities Adam tracks. Its variance estimate includes some drift over training time, so we will check it against a more direct but costlier micro-batch estimate on a subset of steps. We will also test an isotropic estimator, $\Sigma = \bar\sigma^2 I$ with $\bar\sigma^2$ the average coordinate variance, to see whether the diagonal structure matters.

Because $d^\star$ shrinks when noise dominates, it also acts as an implicit step-size control. We will compare two versions: one rescaled to the norm of the average gradient, matching CAGrad's geometry, and one left unscaled.

On the analysis side, we will characterize when the method beats both of its limits on a family of two-task quadratics. In these problems we control the angle between task gradients and the injected noise variance, and expect a crossover governed by a ratio of conflict strength to noise level. This is our analogue of the finite-blocklength question raised by URLLC With Massive MIMO: Analysis and Design at Finite Blocklength: how gradient conflict behaves when sample counts per step are small, rather than in the asymptotic limit.

Baselines

The naive reference point is uniform averaging of task gradients. In the multi-task RL setting this is multi-task SAC, the soft actor-critic reinforcement-learning algorithm with one shared network across tasks. Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning reports 85% success on MT10 and 40% on MT50 for multi-head SAC. The CAGrad paper reports 0.36 on MT50 for multi-task SAC and a 5.59% average per-task drop ($\Delta m\%$) for the MTAN architecture on NYUv2. Every new method must clear this bar, and the comparison isolates whether any combining helps at all.

PCGrad is the closest mechanism, since it is the hard, noise-blind projector our method claims to improve. It reports a CIFAR-100 routing accuracy gain from 67.7% to 77.5% and about 70% on MT50 in its own paper. The CAGrad paper measures it at 0.50 on MT50 and 3.97% $\Delta m$ on NYUv2.

CAGrad is the strongest existing method and the real competitor. The CAGrad paper reports 0.20% $\Delta m$ on NYUv2 and 0.52 success on MT50. Beating it means a lower $\Delta m$ on NYUv2 and higher MT50 success, both outside seed-to-seed variance. No reported result was found on Lacuna for CAGrad on MT10, CityScapes, or Multi-MNIST, so we will run those ourselves. RotoGrad is a secondary baseline on NYUv2 and Multi-MNIST, where its paper gives qualitative claims but no reported number found on Lacuna.

All methods will share architectures (MTAN for NYUv2 and CityScapes, multi-head SAC for Meta-World), data, seeds (at least five; ten for RL), and step counts. Each method's tunable constant gets the same tuning budget on a held-out split: CAGrad's $c$, our normalization choice, and the learning rate for all. Because our method adds a noise estimate, we also report wall-clock time and charge any micro-batch estimation to our compute budget.

Three ablations isolate the key ingredient. Joint zero-forcing ($\Sigma = 0$, pseudo-inverse) tests whether regularization matters at all. A fixed ridge $\Sigma = \lambda I$, with $\lambda$ tuned by grid search, tests whether a well-chosen constant matches adaptive noise estimation; this is the comparison that decides whether "noise-aware" is the contribution. Isotropic versus diagonal $\Sigma$ tests whether the noise structure matters.

Why not just…?

Why not just tune CAGrad's $c$? CAGrad already trades average descent against conflict avoidance, and if one tuned $c$ matches us everywhere, our contribution reduces to a reparameterization. We take this threat seriously. Our claim is that the right trade-off changes with the noise level, which varies with batch size, training phase, and task. The batch-size sweep below settles it. If the best $c$ for CAGrad is stable across batch sizes and tuned CAGrad tracks us at every one, we lose.

Why not just reduce gradient noise first, with bigger batches or smoothed gradients, and then apply PCGrad? If noise is the problem, removing it might make hard projection fine. This is plausible, but larger batches cost compute, and smoothing introduces staleness. We will run PCGrad with an exponential moving average of task gradients, and PCGrad at a batch size enlarged to match our total compute. If either matches us at equal wall-clock time, the MMSE formulation is unnecessary.

Why not just reweight the task losses? Scalar weights change gradient magnitudes but not directions. As Gradient Surgery for Multi-Task Learning notes, positive reweighting cannot remove cancellation between opposed gradients, and RotoGrad: Gradient Homogenization in Multi-Task Learning argues magnitude and direction conflicts are separate problems. We therefore treat loss reweighting as a weaker reference rather than a primary competitor. A magnitude-normalized average serves as that reference so the comparison is not purely rhetorical.

Proposed Experiments

The decisive check is a batch-size sweep on NYUv2 (four batch sizes spanning a factor of eight) and on Meta-World MT10 (varying samples per gradient step). Our method's gain over CAGrad and PCGrad should grow as batches shrink and approach zero at the largest batch. We also log the per-step ratio of $\|\hat G^\top \hat G\|$ to $\mathrm{tr}(\Sigma)$ and check that improvements concentrate in high-noise phases. A flat gain profile across batch sizes would falsify the mechanism, even if average numbers look good.

The second check is the controlled two-task quadratic family. Joint zero-forcing should diverge or stall as the angle between task gradients approaches 180°, averaging should stall under strong conflict, and the MMSE update should track the better of the two across the grid.

The headline comparisons are $\Delta m\%$ on NYUv2 and CityScapes, success rate on MT10 and MT50, and accuracy on Multi-MNIST and the CIFAR-100 routing setup, against the reported figures above where they exist. On MT50 we also report time per step against CAGrad's subsampled fast variant, which its authors report runs 5x faster than PCGrad, and test task subsampling for our method to keep the $K \times K$ solve and the $K$ backward passes affordable.