Auxiliary Task Update Decomposition: the Good, the Bad and the neutral

Lucio M. Dery, Yann N. Dauphin, David Grangier

Overview

While deep learning has proven immensely powerful in data-rich environments, many real-world applications suffer from limited training data. To overcome this, practitioners often turn to multi-task learning (MTL) or pre-training, leveraging auxiliary tasks to boost performance on a primary task. However, this approach is fraught with challenges, most notably the risk of "negative transfer," where auxiliary updates actively interfere with the primary task.

This paper introduces ATTITTUD (Auxiliary Task Training with Influence from Target Task Update Direction), a novel, model-agnostic framework that decomposes auxiliary task gradients into three distinct components based on their impact on the primary task: those that help, those that harm, and those that are neutral. By selectively re-weighting these components, ATTITTUD provides a scalable and effective way to harness auxiliary data without compromising the primary objective.

The Challenge of Negative Transfer

In data-starved scenarios, borrowing statistical regularities from auxiliary tasks is a common strategy. Yet, treating all tasks symmetrically or relying on coarse metrics—like the cosine similarity between average gradients—often fails to prevent negative transfer. Auxiliary tasks do not always align perfectly with the primary task; their gradients can sometimes push the model parameters in detrimental directions. There is a critical need for a fine-grained, primary-task-centric approach that filters and manipulates auxiliary updates to ensure they are strictly beneficial or, at worst, harmless.

Methodology: Decomposing the Gradient

The core innovation of ATTITTUD lies in its geometric decomposition of the auxiliary gradient ($\mathbf{g}_{aux}$) relative to the primary task's gradient subspace. The framework splits the auxiliary update into three orthogonal directions:

  1. The Good ($\mathbf{g}^+_{aux}$): Components that align with the primary task's gradient subspace, actively helping to minimize the primary loss.
  2. The Bad ($\mathbf{g}^-_{aux}$): Components that directly oppose the primary task's gradient, causing interference and negative transfer.
  3. The Neutral ($\mathbf{g}^\perp_{aux}$): Components orthogonal to the primary task's gradient subspace, having no immediate first-order impact on the primary loss.
Decomposition of the auxiliary gradient into good, bad, and neutral components.

The ATTITTUD Algorithm

To make this decomposition computationally feasible for large-scale neural networks, ATTITTUD employs several advanced techniques:

  • Subspace Projection: Rather than relying on a single average primary gradient, the method constructs a subspace spanned by per-example primary gradients (the Jacobian). This captures a richer neighborhood of helpful directions.
  • Randomized SVD: Computing the full Jacobian is prohibitively expensive. ATTITTUD utilizes randomized Singular Value Decomposition (SVD) and efficient Jacobian-vector products to approximate the top-$k$ singular vectors of the primary task's Jacobian.
  • Flexible Weighting: The framework introduces hyperparameters ($\eta$) to independently scale the decomposed components. For instance, practitioners can invert the "Bad" gradients ($\eta_- = -1$) to make them constructive, or retain the "Neutral" components to encourage general feature learning without immediate harm.

Experimental Setup

The authors rigorously evaluated ATTITTUD across diverse domains to demonstrate its versatility:

  • Text Classification: Sentiment analysis using Amazon Helpfulness and IMDb Movie Review datasets, focusing on out-of-distribution auxiliary data.
  • Image Classification: Tasks included MultiCifar100 (treating super-classes as primary) and a severely data-starved "Cat-vs-Dog" subset of CIFAR-10 (only 50 examples per class).
  • Medical Imaging: Transfer learning from ImageNet (auxiliary) to the ChexPert dataset (primary, 5k examples).

Key Findings and Results

ATTITTUD consistently outperformed strong baselines, including vanilla pre-training, standard MTL, and PCGrad:

  • Selfish Optimization: Unlike standard MTL, which seeks a Pareto optimal balance across all tasks, ATTITTUD is intentionally asymmetric, prioritizing the primary task above all else.
  • The Value of Neutral Updates: Surprisingly, retaining "neutral" components ($\mathbf{g}^\perp_{aux}$) often proved beneficial. While they do not immediately reduce the primary loss, they help the model learn generalized features that improve overall robustness.
  • Performance Gains:
    • In text classification (Amazon/IMDb), ATTITTUD achieved an F1 score of 68.5, surpassing TAPT's 67.8.
  • On the low-resource Cat-vs-Dog task, it reached 67.1% accuracy, significantly beating PCGrad (64.2%) and vanilla pre-training (64.5%).
  • In medical imaging (ChexPert), the method achieved an 83.3% AUC, outperforming standard ImageNet pre-training (81.4%).

Limitations and Future Directions

While highly effective, ATTITTUD introduces new hyperparameters, specifically the scaling factors ($\eta$) and the subspace rank ($k$).

Impact of subspace dimensionality on accuracy in high vs. low resource settings.

As illustrated above, the optimal size of the subspace ($k$) varies depending on the data regime. In low-resource settings, a smaller $k$ is preferable to prevent overfitting, whereas high-resource settings can leverage a larger $k$ for more nuanced gradient manipulation. Future work will likely focus on automating the selection of these hyperparameters and further exploring the theoretical underpinnings of using singular vectors for variance reduction in gradient descent.

Summary updated

Abstract

While deep learning has been very beneficial in data-rich settings, tasks with smaller training set often resort to pre-training or multitask learning to leverage data from other tasks. In this case, careful consideration is needed to select tasks and model parameterizations such that updates from the auxiliary tasks actually help the primary task. We seek to alleviate this burden by formulating a model-agnostic framework that performs fine-grained manipulation of the auxiliary task gradients. We propose to decompose auxiliary updates into directions which help, damage or leave the primary task loss unchanged. This allows weighting the update directions differently depending on their impact on the problem of interest. We present a novel and efficient algorithm for that purpose and show its advantage in practice. Our method leverages efficient automatic differentiation procedures and randomized singular value decomposition for scalability. We show that our framework is generic and encompasses some prior work as particular cases. Our approach consistently outperforms strong and widely used baselines when leveraging out-of-distribution data for Text and Image classification tasks.
BibTeX
@inproceedings{dery2021auxiliary,
  author = {Lucio M. Dery and Yann N. Dauphin and David Grangier},
  title = {{Auxiliary Task Update Decomposition: the Good, the Bad and the neutral}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=wivii-2TYf},
}
Figure