Uncertainty Regularized Multi-Task Learning

Kourosh Meshgi, Maryam Sadat Mirzaei, Satoshi SekineACL 2022

TL;DR

This paper introduces a novel multi-task learning (MTL) approach that leverages task uncertainty to improve the generalization of shared features. By regularizing the model to minimize task uncertainty during fine-tuning, they achieve better performance and knowledge transfer across related tasks.

What Problem Does This Paper Address?

Multi-task learning (MTL) aims to improve model performance by training on multiple related tasks simultaneously, leveraging shared information. A common approach is to share parameters, particularly in deep neural networks, to learn a common representation. However, a key challenge is the "fine-tuning vs. generalization" trade-off: aggressively fine-tuning on one task can negatively impact the performance of other tasks or lead to over-generalization of the shared features. Existing methods try to balance this trade-off through various techniques like adjusting task weights or using adversarial learning, but none explicitly focus on using task uncertainty as a direct signal to guide the learning process and prevent detrimental changes to shared features when fine-tuning individual tasks.

Methods

The core of this paper is the introduction of an uncertainty regularization term to the standard multi-task learning loss function. The proposed method, Uncertainty-Regularized Fully-Shared MTL (UFS-MTL), builds upon a fully-shared architecture where a single LSTM layer extracts features for all tasks.

The key innovation is the addition of an uncertainty loss term ($L_{unc}$) that measures the uncertainty of each task's predictions. This uncertainty is calculated based on the model's output probabilities and can be quantified using measures like least confidence, margin, or Shannon's entropy. The total loss then becomes a combination of the standard task loss ($L_{task}$) and this uncertainty loss ($L_{unc}$), controlled by a regularization parameter ($\lambda$).

The intuition is that when fine-tuning a specific task, if this fine-tuning leads to increased uncertainty in other tasks (meaning the shared features are becoming less reliable for them), the uncertainty regularization term will penalize this change. This encourages the model to learn shared features that are not only accurate for the current task but also maintain low uncertainty across all tasks, thus promoting better generalization.

The paper also explores extensions like:

  • Semi-Supervised Learning: The uncertainty loss is label-agnostic, allowing the inclusion of unlabeled data to further improve learning.
  • Fine-Tuning: After initial joint training, individual tasks can be further fine-tuned with the shared layers frozen.
  • Pre-Training: Initializing the shared layers with an unsupervised language model.
  • Auxiliary Task: Incorporating an additional task (like Part-of-Speech Tagging) to enrich the shared feature space.

Datasets

The experiments were conducted on two main settings:

  • Multiple Domains: 16 text classification datasets were used, primarily product and movie review datasets. These datasets were converted to binary positive/negative labels. Examples include datasets from Blitzer et al. (2007), IMDB, and MR.
  • Multiple Tasks: A heterogeneous multi-task learning scenario involving three distinct NLP tasks: Part-of-Speech (POS) tagging, chunking, and Named Entity Recognition (NER). Datasets used were the Wall Street Journal (WSJ) subset of Penn Treebank, CoNLL 2000 (for chunking), and CoNLL 2003 (for NER).

Key Insights and Ideas

  • Uncertainty as a Signal for Generalization: The paper demonstrates that task uncertainty is a valuable signal for guiding MTL. By actively minimizing uncertainty in other tasks during fine-tuning, the model can learn more robust and generalizable shared representations.
  • Balancing Fine-tuning and Generalization: The proposed uncertainty regularization provides a principled way to balance the immediate gains from fine-tuning a task with the long-term benefit of maintaining good performance across all tasks.
  • Shared Features for Unseen Tasks: The experiments on "leave-one-out" suggest that the learned shared representations, regularized by uncertainty, can effectively transfer knowledge to entirely new, unseen tasks.
  • Effectiveness Across Architectures: While the paper uses an LSTM baseline for simplicity, the authors suggest that the uncertainty regularization can be applied to other architectures like RNNs, CNNs, and Transformers.

Key Results

  • Improved Accuracy: The UFS-MTL method consistently improved performance across the 16 text classification tasks, achieving up to a 5.7% improvement on individual tasks and an average improvement of 3.5% over the vanilla FS-MTL baseline. It also outperformed other competitive MTL models.
  • State-of-the-Art Performance: In the multi-task sequence tagging experiment, UFS-MTL achieved state-of-the-art performance, outperforming models like SSP-MTL and Meta-MTL.
  • Knowledge Transfer: In the leave-one-out experiment, UFS-MTL showed strong performance when transferring knowledge to unseen tasks, indicating effective generalization of learned shared features.
  • Effect of Regularization Parameter ($\lambda$): The study found that a moderate $\lambda$ is optimal. Too small a $\lambda$ makes the model behave like FS-MTL, while too large a $\lambda$ can lead to catastrophic forgetting due to over-generalization.

Limitations and Future Work

  • Error Analysis: The authors identified two main sources of errors: complex sentence structures (negation) and sentences requiring external knowledge or analogies. They suggest that auxiliary tasks can help in capturing these nuances.
  • Choice of Uncertainty Measure: The paper explored different uncertainty measures (least confidence, margin, entropy) and found that margin uncertainty (SM) generally performed well. However, the choice of measure might impact performance, and further investigation could be beneficial.
  • Computational Cost: While not explicitly detailed as a limitation, multi-task learning inherently increases computational complexity. The proposed method adds an uncertainty calculation, which could further increase this cost.
  • Future Work: The authors suggest exploring more complex architectures and applying the method to a wider range of tasks and domains. They also highlight the potential of using auxiliary tasks to address specific error types.

Summary updated

BibTeX
@inproceedings{meshgi2022uncertainty,
  author = {Kourosh Meshgi and Maryam Sadat Mirzaei and Satoshi Sekine},
  title = {{Uncertainty Regularized Multi-Task Learning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure