Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models

Zhen Qi, Jiajing Chen, Shuo Wang, Bingying Liu, Hongye Zheng, Chihang Wang

TL;DR

This paper demonstrates that Large Language Models (LLMs) like GPT-4 perform better when trained on multiple tasks simultaneously rather than specializing in just one. By implementing a Multi-Task Learning (MTL) framework with a shared GPT-4 backbone and specialized gradient optimization, the researchers achieved superior results in both text classification and automatic summarization compared to single-task models.

What Problem Does This Paper Address?

In the current NLP landscape, we often treat models as specialists—training one version for sentiment analysis and another for document summarization. However, this "single-task" approach is inefficient and ignores the fact that different language tasks often rely on the same underlying semantic understanding.

The challenge with Multi-Task Learning (MTL) is that tasks can sometimes "fight" each other during training. For instance, the gradient updates required to improve summarization might inadvertently degrade classification performance—a phenomenon known as task conflict. Furthermore, if one task is much harder than the other, the model might become biased. This paper explores how to architect a system that allows tasks to share knowledge effectively without these negative side effects.

Methods

The authors propose a unified framework consisting of three main components:

  1. Shared Feature Extractor: They use a heavy-hitter LLM (GPT-4) as the "backbone" to map input text into a high-dimensional semantic space.
  2. Task-Specific Heads: They attach independent "heads" to this backbone—a classification head for categorical labels and a generation head for producing summaries.
  3. Gradient Optimization: This is the core innovation. To prevent task conflict, the authors use a joint objective function that minimizes the cosine similarity between task gradients. Essentially, they ensure that the "direction" the model learns for Task A doesn't point in the opposite direction of what is needed for Task B.

They also employed a step-by-step training strategy: first pre-training the shared extractor to learn general language features, followed by joint fine-tuning with task-specific modules like LoRA (Low-Rank Adaptation) to keep computational overhead low.

Datasets

The study primarily utilizes the GLUE (General Language Understanding Evaluation) benchmark. GLUE is a collection of diverse natural language tasks, including:

  • CoLA: Linguistic rationality.
  • MNLI: Natural language inference across domains.
  • STS-B: Semantic sentence pair similarity.

By using GLUE, the authors ensured their model was tested against a wide variety of linguistic challenges, ranging from simple sentence classification to complex logical reasoning.

Key Insights and Ideas

  • Synergy over Specialization: The most significant insight is that MTL isn't just a way to save space; it actually improves performance. The shared representation allows the model to "borrow" context from the summarization task to better understand the nuances required for classification.
  • Gradient Alignment is Crucial: Simply throwing two loss functions together usually leads to instability. By explicitly managing the relationship between task gradients (using the cosine similarity approach), the authors maintained a stable loss convergence throughout training.
  • LLMs as Universal Backbones: The results reinforce the idea that models like GPT-4 have reached a level of "rich semantic representation" where they can serve as a foundation for almost any NLP task when paired with the right specialized heads.

Key Results

The proposed MTL model ("Ours") outperformed all baselines, including single-task GPT-4 and the multi-task version of GPT-3.

Model Classification Acc (%) Summarization (ROUGE-1)
Single-task GPT-4 91.5 38.6
GPT-3 Multi-task 88.4 41.2
Ours (GPT-4 MTL) 93.6 44.8

Specifically, the MTL approach provided a 2.1% boost in accuracy and a 6.2 point jump in ROUGE-1 scores over the single-task GPT-4 counterparts. It also significantly outperformed traditional baselines like Bi-LSTM+Attention, which struggled particularly with the generation task.

Limitations and Future Work

While successful, the authors acknowledge several hurdles:

  • Computational Cost: Training and fine-tuning models at the scale of GPT-4 remains prohibitively expensive for many researchers.
  • Static Weighting: While they used gradient alignment, the task weights were largely static. Future work could involve dynamic task adjustment, where the model automatically shifts its focus to the task it is currently struggling with most.
  • Multimodality: The authors suggest that the next frontier for this MTL framework is incorporating multimodal data (images and speech) to move closer toward General Artificial Intelligence (AGI).

Summary updated

Abstract

This study aims to explore the performance improvement method of large language models based on GPT-4 under the multi-task learning framework and conducts experiments on two tasks: text classification and automatic summary generation. Through the combined design of shared feature extractors and task-specific modules, we achieve knowledge-sharing and optimization of multiple tasks in the same model. The experiment uses multiple subtasks of the GLUE dataset to compare the performance of the multi-task model with the single-task GPT-4, the multi-task version of GPT-3, the BERT basic model, and the classic Bi-LSTM with Attention model. The results show that the proposed multi-task learning model outperforms other comparison models in terms of text classification accuracy and ROUGE value of summary generation, demonstrating the advantages of multi-task learning in improving model generalization ability and collaborative learning between tasks. The model maintains a stable loss convergence rate during training, showing good learning efficiency and adaptability to the test set. This study verifies the applicability of the multi-task learning framework in large language models, especially in improving the model's ability to balance different tasks. In the future, with the combination of large language models and multimodal data and the application of dynamic task adjustment technology, the framework based on multi-task learning is expected to play a greater role in practical applications across fields and provide new ideas for the development of general artificial intelligence.
BibTeX
@misc{qi2024optimizing,
  author = {Zhen Qi and Jiajing Chen and Shuo Wang and Bingying Liu and Hongye Zheng and Chihang Wang},
  title = {{Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models}},
  year = {2024},
  eprint = {2412.06249},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.06249},
}
Figure