Multi-Task Learning with Multi-Task Optimization

Lu Bai, Abhishek Gupta, Yew-Soon OngarXiv 2024 · arXiv · OpenReview

TL;DR

This paper introduces MT²O, a novel algorithm for Multi-Task Learning (MTL) that tackles conflicting task objectives by framing it as a Multi-Objective Optimization (MOO) problem. MT²O decomposes MOO into multiple subproblems and solves them jointly using a new gradient descent method that iteratively transfers model parameters between subproblems, leading to faster convergence and a better set of Pareto-optimal solutions.

What Problem Does This Paper Address?

Traditional Multi-Task Learning (MTL) aims to improve model performance by training on multiple related tasks simultaneously. However, these tasks can sometimes conflict, meaning improving one task's performance might degrade another's. In such scenarios, a single "best" model is often insufficient, as users might need a range of models that represent different trade-offs between task performances. Existing MTL methods often struggle to find this diverse set of optimal solutions, especially when task conflicts are present. This paper addresses the challenge of efficiently discovering a representative set of Pareto-optimal models that capture these various trade-offs in a single training pass.

Methods

The core of the proposed method, MT²O (Multi-Task Learning with Multi-Task Optimization), lies in reframing MTL as a Multi-Objective Optimization (MOO) problem. This MOO problem is then decomposed into a set of simpler, scalar-valued subproblems. The novelty comes from how these subproblems are solved:

  • Decomposition: The MOO problem is broken down using either weighted-sum or a smoothed Tchebycheff scalarization approach. This creates multiple subproblems, each aiming for a specific trade-off.
  • Multi-Task Gradient Descent (MTGD): A novel gradient descent algorithm is introduced to solve these subproblems jointly.
  • Iterative Parameter Transfer: The key innovation is the iterative transfer of model parameters between these subproblems during optimization. This "knowledge transfer" leverages the intuition that solutions to similar subproblems (defined by similar trade-off preferences) are likely to be related. This transfer is managed by a dynamically updated transfer matrix.
  • Theoretical Guarantees: The paper provides a theorem demonstrating that this parameter transfer mechanism can accelerate convergence compared to solving subproblems independently, especially under certain conditions (e.g., strongly convex objectives, symmetric transfer).

Datasets

The paper evaluates MT²O on a variety of datasets:

  • Synthetic MOO Problems: P1 (concave Pareto front), ZDT1, and ZDT2 (standard MOO benchmarks).
  • Multi-Task Learning Datasets:
    • MultiMNIST, MultiFashionMNIST, and Multi-(Fashion+MNIST): These datasets involve classifying digits or clothing items from combined images, creating two distinct classification tasks.
  • CelebA: Used for multi-attribute binary classification (17 attributes), representing a scenario with many tasks.
  • River Flow dataset: An 8-task multi-target regression problem.
  • NYUv2 dataset: A challenging scene understanding dataset involving semantic segmentation, depth estimation, and surface normal estimation (three tasks).

Key Insights and Ideas

  • Pareto Front as a Richer Solution: The paper strongly advocates for viewing the Pareto front of solutions as a more informative outcome than a single optimal model, especially in real-world scenarios with conflicting objectives.
  • Leveraging Subproblem Similarities: The core hypothesis is that by solving related subproblems (representing similar trade-offs) simultaneously, information can be shared to speed up convergence. This is analogous to how transfer learning works but applied across different optimization objectives within the same model.
  • Parameter Transfer as a Convergence Accelerator: The theoretical analysis and experimental results suggest that the iterative transfer of model parameters is not just a mechanism for diversity but a genuine accelerator for finding the Pareto front.
  • Bridging Gradient-Based and Evolutionary MOO: The work aims to bring the benefits of population-based evolutionary algorithms (which naturally explore diverse solutions) to gradient-based deep learning methods for MOO.

Key Results

  • State-of-the-Art Performance: MT²O consistently outperforms or matches existing state-of-the-art methods in discovering well-distributed Pareto-optimal sets across various synthetic and real-world datasets.
  • Faster Convergence: On the large NYUv2 dataset, MT²O achieved nearly two times faster convergence in terms of hypervolume (HV) compared to the next-best method. Table I also shows significant HV improvements with transfer enabled, especially at early iterations.
  • Effective for Many Tasks: The method demonstrates strong performance even when dealing with a large number of tasks (e.g., 17 attributes on CelebA).
  • Robustness: While some methods showed high variance, MT²O generally provided more stable results.

Limitations and Future Work

  • Hyperparameter Tuning: The paper acknowledges that hyperparameter tuning (e.g., for To, J, learning rate a, and smoothing parameters as, ε) can be crucial for optimal performance, and extensive tuning was omitted to focus on the core algorithm.
  • Transfer Mechanism Sophistication: While the current parameter transfer is scalar, the authors suggest exploring more sophisticated transfer mechanisms, such as element-wise or adaptive transfer coefficients, or even transferring only specific layers (e.g., convolutional layers).
  • Scalability of Reference Vectors: For problems with many tasks, selecting an appropriate set of reference vectors for decomposition can be challenging. The paper explored using unit vectors for the CelebA dataset as a simplification.
  • Future Work: The authors encourage further research into integrating multitasking with gradient-based optimization for one-pass learning of specialized models and exploring more advanced transfer strategies.

Summary updated

Abstract

Multi-task learning solves multiple correlated tasks. However, conflicts may exist between them. In such circumstances, a single solution can rarely optimize all the tasks, leading to performance trade-offs. To arrive at a set of optimized yet well-distributed models that collectively embody different trade-offs in one algorithmic pass, this paper proposes to view Pareto multi-task learning through the lens of multi-task optimization. Multi-task learning is first cast as a multi-objective optimization problem, which is then decomposed into a diverse set of unconstrained scalar-valued subproblems. These subproblems are solved jointly using a novel multi-task gradient descent method, whose uniqueness lies in the iterative transfer of model parameters among the subproblems during the course of optimization. A theorem proving faster convergence through the inclusion of such transfers is presented. We investigate the proposed multi-task learning with multi-task optimization for solving various problem settings including image classification, scene understanding, and multi-target regression. Comprehensive experiments confirm that the proposed method significantly advances the state-of-the-art in discovering sets of Pareto-optimized models. Notably, on the large image dataset we tested on, namely NYUv2, the hypervolume convergence achieved by our method was found to be nearly two times faster than the next-best among the state-of-the-art.
BibTeX
@misc{bai2024multi,
  author = {Lu Bai and Abhishek Gupta and Yew-Soon Ong},
  title = {{Multi-Task Learning with Multi-Task Optimization}},
  year = {2024},
  eprint = {2403.16162},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.16162},
}
Figure