Multi-Objective Deep Learning with Adaptive Reference Vectors

Weiyu Chen, James KwokNeurIPS 2022 · OpenReview

TL;DR

Deep learning often requires balancing conflicting goals, like accuracy versus fairness or performance across multiple tasks. This paper introduces GMOOAR, an algorithm that automatically learns the best "reference vectors" to map out the entire trade-off landscape (the Pareto front) during training. By treating the search for these trade-offs as a bilevel optimization problem, the authors achieve more diverse and higher-quality solutions than previous methods with minimal computational overhead.

What Problem Does This Paper Address?

In Multi-Objective Optimization (MOO), there isn't a single "best" solution; instead, there is a Pareto front (PF)—a set of solutions where you cannot improve one objective without hurting another. To find these solutions, researchers often use Reference Vectors (RVs) to guide the model toward specific regions of the PF.

Comparison of Fixed vs. Adaptive Reference Vectors

The challenge is that most current methods use fixed reference vectors, usually spread out uniformly. However, the actual shape of the Pareto front is unknown before training. If the front is "irregular" (e.g., very curved or disconnected), fixed vectors might point to non-existent parts of the front, leading to solutions that are clumped together or miss entire regions of the trade-off space. This leaves decision-makers with a poor understanding of the available choices.

Methods

The authors propose GMOOAR (Gradient-based Multi-Objective Optimization with Adaptive Reference vectors). The core innovation is formulating the problem as bilevel optimization:

  1. Lower-level problem: Optimize the model parameters (the neural network) to find the best solutions for a given set of reference vectors.
  2. Upper-level problem: Adjust the reference vectors themselves to maximize the quality of the resulting solution set.

To measure "quality," they use two different functions:

  • Uniformity (GMOOAR-U): Uses a Gaussian kernel to push solutions away from each other, ensuring they are spread evenly across the objective space.
  • Hypervolume (GMOOAR-HV): Optimizes for the total "volume" covered by the solutions in the objective space, a standard metric for both convergence and diversity.

To keep the algorithm efficient enough for deep learning, they use a conditioned neural network (based on the YOTO architecture). This allows a single model to represent multiple points on the Pareto front simultaneously by taking the reference vector as an additional input. They also use a first-order approximation for the bilevel gradient, avoiding the massive computational cost of calculating second-order derivatives (Hessians).

Datasets

The researchers evaluated GMOOAR across several domains:

  • Synthetic Benchmarks: DTLZ2 and MaF1 (standard problems used to test MOO algorithms).
  • Multi-task Learning: Multi-MNIST, Multi-Fashion, and a hybrid Multi-Fashion+MNIST dataset.
  • Accuracy-Fairness Trade-offs: Three tabular datasets—Adult, Compass, and Default—where the goal is to balance prediction accuracy with demographic fairness.
  • Large-scale Vision: CelebA (40 facial attribute tasks) using the EfficientNet-B4 architecture to prove scalability.

Key Insights and Ideas

  • Fixed Vectors are a Blind Guess: The paper visually demonstrates that uniform reference vectors do not lead to uniform solutions. By making vectors adaptive, the model "discovers" the shape of the Pareto front and populates it more effectively.
  • Bilevel Optimization Doesn't Have to Be Slow: By using a first-order "inexpensive" solver, the authors show that adapting reference vectors adds negligible time compared to standard training.
  • Handling Irregularity: GMOOAR shines particularly on "irregular" Pareto fronts where traditional methods like COSMOS or EPO fail to provide a diverse set of solutions.
  • Conditioned Networks vs. Hypernetworks: While hypernetworks (networks that generate other networks) are popular for MOO, they are often 100x larger than the base model. GMOOAR’s use of conditioned networks achieves similar results with only about 37% parameter overhead.

Key Results

  • Superior Diversity: In multi-task learning (Multi-MNIST), GMOOAR-U produced significantly more uniform solution sets than state-of-the-art baselines.
  • Higher Hypervolume: Across almost all tests, GMOOAR-HV achieved the highest hypervolume scores, indicating it finds solutions that are both closer to the true Pareto front and better distributed.
Multi-task learning results comparing GMOOAR to baselines
  • Scalability: The authors successfully ran GMOOAR on EfficientNet-B4 (17 million parameters). Many competing gradient-based MOO methods (like PHN or MOO-SVGD) were unable to run on this scale due to memory or computational constraints.
  • Efficiency: GMOOAR-U is highly scalable; its runtime remains relatively flat even as the number of tasks increases, whereas Hypervolume-based methods (including their own GMOOAR-HV) slow down because HV calculation is NP-hard.

Limitations and Future Work

  • The HV Bottleneck: While GMOOAR-U (uniformity) scales well, GMOOAR-HV (hypervolume) becomes computationally expensive as the number of objectives increases beyond 10-15.
  • Discrete vs. Continuous: The current method finds a discrete set of $n$ solutions. The authors suggest that future work could incorporate exploration directions to generate a truly continuous Pareto front.
  • Proxy Quality Functions: The "uniformity" measure is a proxy; finding even better upper-level objectives that capture "quality" without the computational cost of Hypervolume remains an open area.

Summary updated

Abstract

Many deep learning models involve optimizing multiple objectives. Since objectives are often conflicting, we aim to get diverse and representative trade-off solutions among these objectives. Gradient-based multi-objective optimization (MOO) algorithms using reference vectors have shown promising performance. However, they may still produce undesirable solutions due to mismatch between the pre-specified reference vectors and the problem's underlying Pareto front. In this paper, we propose a novel gradient-based MOO algorithm with adaptive reference vectors. We formulate reference vector adaption as a bilevel optimization problem, and solve it with an efficient solver. Theoretical convergence analysis is also provided. Experiments on an extensive set of learning scenarios demonstrate the superiority of the proposed algorithm over the state-of-the-art.
BibTeX
@inproceedings{chen2022multi,
  author = {Weiyu Chen and James Kwok},
  title = {{Multi-Objective Deep Learning with Adaptive Reference Vectors}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=9-8YT5G36g-},
}
Figure