Task Uncertainty Loss Reduce Negative Transfer in Asymmetric Multi-task Feature Learning (Student Abstract)

Rafael Peres da Silva, Chayaporn Suphavilai, Niranjan NagarajanAAAI 2021 · DOI

Introduction

Multi-task learning (MTL) is a powerful paradigm in machine learning that leverages shared information across related tasks to improve overall model performance. The underlying assumption is that tasks share common features; for instance, learning to identify "cats" should theoretically assist in identifying "dogs" due to shared visual characteristics like fur or ears. However, this promise is often hindered by Negative Transfer (NT)—a phenomenon where unrelated or noisy tasks interfere with each other, leading to degraded performance compared to training tasks individually (Single-Task Learning, or STL).

The Challenge of Identifying Task Reliability

Existing approaches to mitigate NT, such as Asymmetric Multi-task Feature Learning (AMTFL), attempt to weight tasks based on their "reliability." Traditionally, reliability is measured by training loss: tasks with lower loss are given more influence over the shared feature representation.

However, this metric can be deceptive. A low training loss might simply indicate that a model is overfitting to a small or noisy dataset, rather than capturing meaningful, transferable features. The challenge lies in accurately identifying true task reliability to ensure that knowledge transfer is beneficial rather than detrimental.

Methodology: U-AMTFL

To address this, the authors propose Uncertainty-weighted Asymmetric Multi-task Feature Learning (U-AMTFL). This method shifts the focus from raw loss values to aleatoric homoscedastic uncertainty—the inherent noise specific to a task that remains constant across different inputs.

In the U-AMTFL framework, task-specific observation noise ($\sigma_t$) is treated as a learnable parameter within the neural network.

  • Classification: The loss is adjusted using a softmax function scaled by the uncertainty.
  • Regression: The loss function is modified to include a term that penalizes high error while balancing it against the log of the uncertainty ($\log \sigma_t$).

By making uncertainty a learnable part of the network, the model automatically learns to "down-weight" tasks that are inherently noisy or difficult. This prevents them from corrupting the shared feature representations used by other tasks, effectively creating an asymmetric "teacher-student" relationship where reliable tasks guide the learning of others without being polluted by their noise.

Experimental Evaluation

The researchers evaluated U-AMTFL on two distinct datasets representing different challenges:

  1. MNIST-Imbalanced: A variant of the handwritten digit dataset where digits 8 and 9 have significantly fewer samples, making them "harder" tasks prone to negative transfer.
  2. GDSC (Genomics of Drug Sensitivity in Cancer): A high-stakes pharmacogenomics dataset involving 805 cancer cell lines and 181 drugs. This dataset is characterized by small sample sizes per task, making it a prime candidate for testing robustness against NT.

Key Results

Performance on MNIST-Imbalanced

On the MNIST-Imbalanced dataset, U-AMTFL demonstrated superior performance over standard CNNs, MTL-CNNs, and the previous state-of-the-art Deep-AMTFL. As shown in Figure 1, it achieved the lowest mean classification error and provided the most significant accuracy improvements, particularly on the most challenging tasks (digits 8 and 9). Notably, U-AMTFL was the only model that did not suffer from negative transfer on these sparse tasks.

Mean classification error and per-task accuracy improvements on MNIST-Imbalanced.

Robustness in Pharmacogenomics

In the GDSC experiments, U-AMTFL again outperformed its peers. While all MTL methods showed average improvements over STL, U-AMTFL achieved the best overall Mean Squared Error (1.7342 vs. STL's 2.6210). Crucially, as illustrated in Figure 2, U-AMTFL significantly reduced the incidence of negative transfer. It recorded only 2 cases of negative transfer out of 181 drugs, whereas other methods saw significantly more instances where the multi-task approach performed worse than a simple linear regression.

Comparison of model error relative to STL and the number of negative transfer tasks across different models.

Conclusion and Future Work

U-AMTFL provides a robust mechanism for shielding multi-task models from the harmful effects of negative transfer by using learnable task uncertainty as a weighting factor. This approach is particularly valuable in data-scarce domains like bioinformatics, where traditional loss-based weighting may fail due to overfitting.

Future research will explore incorporating feature-space uncertainty and investigating the specific task relationships that drive uncertainty in complex biological datasets. Additionally, the authors aim to test the scalability of this learnable uncertainty parameter to thousands of simultaneous tasks.

Summary updated

BibTeX
@inproceedings{silva2021task,
  author = {Rafael Peres da Silva and Chayaporn Suphavilai and Niranjan Nagarajan},
  title = {{Task Uncertainty Loss Reduce Negative Transfer in Asymmetric Multi-task Feature Learning (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17930},
}
Figure