Multi-Objective Bilevel Learning

Zhiyao Zhang, Zhuqing Liu, Xin Zhang, Wen-Yen Chen, Jiyan Yang, Jia LiuarXiv 2025 · DOI · arXiv

Introduction

Modern machine learning is increasingly characterized by the need to optimize multiple conflicting objectives within nested decision-making structures. This class of problems, known as Multi-Objective Bilevel Learning (MOBL), is foundational to tasks such as meta-learning, hyperparameter optimization, and robust training under data corruption. In a typical MOBL setting, an "outer" loop optimizes a set of objectives that depend on the optimal parameters found in an "inner" loop task.

The primary technical challenge in MOBL arises from the accumulation of errors. Because the inner loop is rarely solved to global optimality, the resulting hypergradients used to update the outer loop are inherently biased. This bias complicates the search for a Pareto-stationary point—a solution where no single objective can be improved without degrading another—particularly when specific user-defined preferences must be strictly enforced.

Methodology: The WC-MHGD Framework

To address these challenges, we present Weighted Chebyshev Multi-Hyper-Gradient-Descent (WC-MHGD). This framework provides the first finite-time convergence rates for preference-guided MOBL in both deterministic and stochastic environments.

The core innovation of WC-MHGD is a dynamic weighting subproblem solved at each iteration. This subproblem balances two critical components:

  1. Stationarity: A term that drives the model toward the Pareto front of optimal trade-offs.
  2. Preference Alignment: A term utilizing Weighted Chebyshev scalarization to ensure the final solution adheres to a user-provided preference vector (e.g., prioritizing accuracy over computational speed).

By adjusting a tunable parameter $u$ within the subproblem, the algorithm can transition between general Pareto exploration and strict adherence to specific preference vectors. To manage the computational complexity of second-order derivatives (Hessians) required for bilevel optimization, the framework employs hypergradient estimation techniques such as Conjugate Gradient (CG) or Neumann Series (NS) approximations, achieving a state-of-the-art convergence rate of $O(1/K)$.

Experimental Evaluation

The framework was rigorously evaluated on two benchmarks: meta-learning using the FC-100 (Fewshot-CIFAR-100) dataset and data hyper-cleaning on MNIST. In the meta-learning scenario, five distinct learning tasks were treated as potentially conflicting objectives to identify a robust shared model initialization.

Comparison of WC-MHGD against baselines MOML and FORUM across five objectives, demonstrating superior coverage and overall performance.

As illustrated in the radar charts, WC-MHGD significantly outperforms existing baselines such as MOML and FORUM. The red boundary of WC-MHGD encompasses a larger area across the five objectives, indicating a more effective discovery of the Pareto front.

Furthermore, the algorithm's ability to respect specific user preferences was validated. By varying the preference vector, the model successfully shifted its performance focus toward the "preferred" objectives while maintaining overall stability across the task suite.

Visualization of preference-guided optimization, showing how WC-MHGD aligns solutions with specific user-defined preferences ranging from moderate to extreme.

Conclusion and Future Work

WC-MHGD establishes a new theoretical and empirical benchmark for MOBL by providing $O(1/K)$ convergence guarantees and robust preference alignment. While the current theoretical framework assumes a strongly convex lower-level objective, future research will focus on extending these guarantees to non-convex inner loops common in deep learning. Additionally, scaling the framework to handle hundreds of objectives and applying it to decentralized settings like federated learning remain high-impact directions for the field.

Summary updated

Abstract

As machine learning (ML) applications grow increasingly complex in recent years, modern ML frameworks often need to address multiple potentially conflicting objectives with coupled decision variables across different layers. This creates a compelling need for multi-objective bilevel learning (MOBL). So far, however, the field of MOBL remains in its infancy and many important problems remain under-explored. This motivates us to fill this gap and systematically investigate the theoretical and algorithmic foundation of MOBL. Specifically, we consider MOBL problems with multiple conflicting objectives guided by preferences at the upper-level subproblem, where part of the inputs depend on the optimal solution of the lower-level subproblem. Our goal is to develop efficient MOBL optimization algorithms to (1) identify a preference-guided Pareto-stationary solution with low oracle complexity; and (2) enable systematic Pareto front exploration. To this end, we propose a unifying algorithmic framework called weighted-Chebyshev multi-hyper-gradient-descent (WC-MHGD) for both deterministic and stochastic settings with finite-time Pareto-stationarity convergence rate guarantees, which not only implies low oracle complexity but also induces systematic Pareto front exploration. We further conduct extensive experiments to confirm our theoretical results.
BibTeX
@misc{zhang2025multi,
  author = {Zhiyao Zhang and Zhuqing Liu and Xin Zhang and Wen-Yen Chen and Jiyan Yang and Jia Liu},
  title = {{Multi-Objective Bilevel Learning}},
  year = {2025},
  eprint = {2511.07824},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.07824},
}
Figure