Investigating Composite Modeling in Multi-Objective Bayesian Optimization for Scientific Design

Ricky Ye, Sahas Goli, Muhammad Ali, Poompol Buathong, Peter I. FrazierarXiv 2026 · arXiv

In scientific research, designing a new material or optimizing a chemical reaction is rarely a one-dimensional task. You aren't just looking for the strongest alloy; you want the strongest alloy that is also lightweight and cheap to produce. This is the domain of Multi-Objective Bayesian Optimization (MOBO).

A new paper from researchers at Cornell University, "Investigating Composite Modeling in Multi-Objective Bayesian Optimization for Scientific Design," dives into a powerful but under-explored strategy: looking "under the hood" of our objective functions to improve optimization efficiency.

TL;DR

This paper demonstrates that modeling intermediate physical responses (Composite modeling) instead of just the final objectives (Black-box modeling) significantly boosts sampling efficiency in multi-objective optimization. The researchers show, both theoretically and experimentally, that these gains are most dramatic when the relationship between intermediate results and final goals is complex—specifically, when it is non-bijective and non-affine.

What Problem Does This Paper Address?

In many scientific fields, we don't just measure a final "score." Instead, we measure intermediate responses that are then plugged into known formulas. For example, in color formulation, a lab might measure light reflectance (intermediate) and then use a standard formula to calculate how close that color is to a target (objective).

Standard MOBO often treats the entire process as a "black box," modeling only the final objective. While "Composite modeling"—which builds models for the intermediate steps—has shown promise in single-objective tasks, its utility in the more complex multi-objective setting was poorly understood. The authors set out to answer: When does modeling these intermediates actually help, and by how much?

Methods

The researchers compared three primary inference approaches:

  1. Composite: Places a Gaussian Process (GP) prior on the intermediate responses and conditions on those observations.
  2. BlackBox-GP: The standard approach; places a GP directly on the final objectives.
  3. BlackBox-Induced: A theoretical "middle ground" that uses the same prior as the Composite model but only allows the algorithm to see the final objective values.

They tested these approaches using four state-of-the-art MOBO methods, including LogEHVI (Log Expected Hypervolume Improvement) and MORBO (Multi-Objective Regional Bayesian Optimization), across a variety of synthetic and real-world scientific benchmarks.

Datasets

The study employed a diverse set of benchmarks:

  • Controlled Synthetic Experiments: 16 different mathematical mappings (bijective vs. non-bijective) to isolate specific functional properties.
  • Scientific-Design Benchmarks: Eight high-fidelity simulations including:
  • Color formulation: Matching fabric colors under different lights.
  • Truss design: Balancing weight vs. displacement in structural engineering.
  • Shallow-lake management: Balancing economic utility vs. phosphorus pollution.
  • Nanoparticle design: Optimizing light scattering for specific color bands.
  • Reaction optimization: Maximizing yield while minimizing waste in chemical synthesis.

Key Insights and Ideas

  • The Bijectivity Rule: The authors proved that if the mapping from intermediates to objectives is bijective (one-to-one) or affine (linear), observing the intermediates provides no additional predictive information over just seeing the final objective, provided you use the same prior.
  • Information Loss in Black Boxes: When a mapping is non-bijective (e.g., a squaring function $h^2$ or a cosine $\cos h$), the final objective "discards" information about the intermediate state. Composite modeling excels here because it preserves and learns from that "lost" data.
  • Kernel Richness: Even in linear (affine) cases, Composite modeling can help in practice. By modeling each intermediate response with its own GP, the algorithm effectively creates a much richer and more flexible "induced" kernel for the final objective than a standard black-box GP could ever fit.

Key Results

The experimental results strongly backed the theory. In controlled synthetic tests, Composite modeling achieved a staggering 99.8% reduction in regret for non-bijective single-objective problems compared to the black-box approach.

In the multi-objective scientific benchmarks, the gains were consistent and significant:

  • Color Formulation: Saw hypervolume gains of 8.7% to 13.8%.
  • Lake Management: Showed gains of 4.1% to 8.7%.
  • Consistency: Composite modeling provided a positive mean final-hypervolume gain in 15 of the 16 experimental comparisons.
Table 1: Controlled synthetic experiments. Final regret reductions (single-objective) and final hypervolume gains (multi-objective) for Composite relative to BlackBox-GP, both in percentage (%) (mean ± SE over 20 paired runs). Positive values favor Composite. Bold marks unadjusted p < 0.05 (one-sided paired t-tests for improvement).
Table 2: Scientific-design experiments. Final hypervolume gains (%, mean ± SE) for Composite relative to BlackBox-GP over 10 paired trials per benchmark (80 for Synthetic 1). Positive values favor Composite. Bold marks unadjusted p < 0.05 (one-sided paired t-tests for improvement). Affine marks affine outer mappings.

Limitations and Future Work

While the benefits are clear, the authors noted a few caveats:

  • Independence Assumption: This study modeled intermediate responses as independent GPs. In reality, these responses are often correlated. Modeling these dependencies could unlock even further gains.
  • Budget Constraints: In some high-dimensional problems (like Nanoparticle design), the gains were modest. The authors suggest that limited evaluation budgets might prevent the composite models from fully realizing their advantage.
  • Future Directions: The researchers recommend exploring how the choice of kernels and the ratio of input dimensions to evaluation budget influence these gains.

For engineers and scientists using BO, the message is clear: if you know the internal structure of your problem, don't hide it. Modeling those intermediate steps is a powerful way to get better designs with fewer experiments.

Summary updated

Abstract

Scientific design often requires balancing competing objectives that are expensive to evaluate through experiments or simulations. Multi-objective Bayesian optimization (MOBO) addresses this challenge by using probabilistic models to guide the selection of promising designs. In many applications, tunable inputs map to observable intermediate responses, which are then combined through known functions to compute the final objectives. Prior work on Bayesian optimization of composite functions shows that learning from these intermediate responses, which we call Composite modeling, can substantially improve sampling efficiency in single-objective optimization, compared to methods that condition only on black-box observations of the objective function. The benefits of Composite modeling in MOBO, however, remain understudied. In this work, we study the benefits of Composite modeling in MOBO and investigate the factors driving variation in its gains. We first establish single-objective conditions under which intermediate observations provide no additional predictive information beyond objective values, where composite and black box methods use the same warped Gaussian process prior. Controlled numerical experiments with versions of four MOBO methods across eight synthetic and application-inspired scientific-design benchmarks examine how these conditions relate to optimization gains in the multi-objective setting, where the black box approach models the objective functions with a standard multi-output Gaussian process. Both investigations support the claim that composite modeling tends to help more when the function mapping intermediate responses to final objectives is non-bijective and non-affine.
BibTeX
@misc{ye2026investigating,
  author = {Ricky Ye and Sahas Goli and Muhammad Ali and Poompol Buathong and Peter I. Frazier},
  title = {{Investigating Composite Modeling in Multi-Objective Bayesian Optimization for Scientific Design}},
  year = {2026},
  eprint = {2611.13565},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13565},
}
Figure