Decoupled Design of Experiments for Expensive Multi-objective Problems

Mickaël Binois, Jürgen Branke, Jonathan E. Fieldsend, Robin C. PurshouseLION 2024 · DOI

TL;DR

This paper introduces a novel "decoupled" approach to designing experiments for expensive multi-objective optimization problems. By strategically choosing which objectives to evaluate at each experimental point, they demonstrate significant improvements in Pareto front estimation, especially when objectives have different evaluation costs or landscape characteristics.

What Problem Does This Paper Address?

In many real-world optimization scenarios, evaluating each objective function can be computationally expensive. When dealing with multiple objectives, traditional methods often evaluate all objectives simultaneously at each experimental point. This can be inefficient, especially if objectives have different costs or behave very differently (e.g., one is smooth, another is highly multimodal). The authors address the challenge of designing experiments for such scenarios, aiming to maximize the information gained per evaluation and improve the accuracy of the estimated Pareto front, which is crucial for decision-making in multi-objective optimization.

Methods

The core of the paper is the proposal of a "decoupled" design of experiments (DoE) strategy. Instead of evaluating all objectives at every point, this approach allows for selective evaluation. The authors propose an iterative method that aims to minimize the integrated mean squared prediction error (IMSPE) of Gaussian process (GP) surrogate models. This involves strategically deciding which objective(s) to evaluate at a given point to maximally reduce the overall prediction error. They compare this to coupled designs (where all objectives are evaluated together) and naive decoupled designs (where objectives are evaluated at different points but without cost or landscape considerations).

Datasets

Not discussed in the paper. The authors use synthetic Gaussian process models to simulate the expensive objective functions, allowing for controlled experiments and avoiding model mismatch issues.

Key Insights and Ideas

  • Decoupling pays off when costs differ: The most significant gains from decoupling are observed when objectives have different evaluation costs. The strategy allows more evaluations of cheaper objectives, leading to better overall performance.
  • Landscape differences matter too: Even with equal costs, if objectives have different landscape characteristics (e.g., smoothness), decoupling can still be beneficial by allowing more focused exploration of the more complex objective.
  • Iterative IMSPE minimization is powerful: Even in the simplest case (equal costs and lengthscales), an iterative approach that minimizes IMSPE (whether coupled or decoupled) outperforms standard Latin Hypercube Sampling (LHS) designs. This suggests that intelligent sequential design is more effective than static space-filling designs.
  • Naïve decoupling can be detrimental: Simply evaluating objectives at different points without considering costs or landscape characteristics can lead to worse results than coupled designs.

Key Results

  • When objective evaluation costs differ, the decoupled approach significantly outperforms coupled designs in terms of IMSPE, RMSE, and Vorob'ev deviation. For instance, when one objective is 10 times more expensive, decoupling leads to substantial improvements.
  • Even when costs and landscape characteristics are the same, the proposed iterative decoupled IMSPE-minimizing approach yields better Pareto front estimates than standard LHS designs.
  • The paper demonstrates that the choice of design strategy (coupled vs. decoupled) can impact the accuracy of the Pareto front estimation, with decoupled strategies showing advantages under specific conditions.

Limitations and Future Work

  • Two objectives only: The current work focuses on two objectives. The authors hypothesize that the benefits of decoupling would extend to more objectives, but this requires further experimental confirmation.
  • GP-generated functions: The experiments were conducted using GP-generated functions to avoid model mismatch. Future work should confirm these results on real-world functions.
  • Learning cost landscape: The current work assumes known costs or costs that are learned incrementally. Investigating methods to learn the cost landscape "on the fly" (i.e., adaptively based on evaluated points) is a promising future direction.
  • Inter-objective dependencies: The paper primarily explores scenarios where objectives can be decoupled. Future research could explore more sophisticated strategies that consider posterior information of one objective when deciding where to evaluate another.

Summary updated

BibTeX
@inproceedings{binois2024decoupled,
  author = {Mickaël Binois and Jürgen Branke and Jonathan E. Fieldsend and Robin C. Purshouse},
  title = {{Decoupled Design of Experiments for Expensive Multi-objective Problems}},
  booktitle = {LION},
  year = {2024},
  doi = {10.1007/978-3-031-75623-8\_4},
}
Figure