Decoupled Function Evaluation in Multi-Objective Optimization

80 concepts40 papers1 proposal

In real-world engineering, we rarely optimize a single metric in isolation. When designing a deep neural network, we must balance predictive accuracy against hardware energy consumption. When planning a drone's trajectory, we must minimize travel time while strictly satisfying physical feasibility constraints.

Traditional multi-objective optimization (MOO) and constrained optimization frameworks treat these metrics as a package deal. To evaluate a candidate design $x$, the system queries an oracle that returns all objectives and constraints simultaneously.

However, this coupled assumption introduces a massive synchronization bottleneck. If objective $A$ takes seconds to compute (e.g., calculating hardware area) while objective $B$ takes hours (e.g., training a model to convergence), a coupled optimizer wastes immense resources. It forces the system to wait for the slowest measurement before making any progress. Worse, it spends expensive compute evaluating a slow metric on a candidate that could have been easily rejected based on a cheap one. As noted by Md Shahriar Iqbal et al. in FlexiBO: A Decoupled Cost-Aware Multi-objective Optimization Approach for Deep Neural Networks (Abstract Reprint), this rigid coupling mandates performing expensive measurements even when a candidate offers minimal information gain.

The Core Concept: Decoupling What from Where

The core insight of decoupled function evaluation is to treat each objective and constraint as an independent information source. Instead of asking the optimizer "Where should we sample next?", we ask a two-part question: "Where should we sample, and which specific metric should we evaluate?"

This paradigm shift allows for heterotopic sampling—evaluating different objectives at entirely different locations in the design space. For instance, Sk Ahnaf Akif Alvi et al. leverage heterotopic sampling in Hierarchical Gaussian Process-Based Bayesian Optimization for Materials Discovery in High Entropy Alloy Spaces to strategically query cheap material properties to inform expensive ones, maximizing the information gained per unit of experimental cost.

By breaking the synchronization barrier, the optimization process can run asynchronously. Fast-to-evaluate metrics can be sampled at high frequency to rapidly map out constraints or eliminate unpromising regions, while expensive evaluations are reserved strictly for high-value candidates.

Why It Works: Additive Information Gain

To understand why decoupled evaluation works mathematically, we look at information-theoretic acquisition functions. In Bayesian optimization, our goal is often to maximize the information gained about the optimal solution set (such as the Pareto frontier or the constrained global optimum).

A key mathematical breakthrough came from realizing that total information gain can be decomposed into a sum of independent, objective-specific components. In Predictive Entropy Search for Multi-objective Bayesian Optimization, Daniel Hernández-Lobato et al. demonstrated that the information provided about the Pareto set scales additively across individual objectives. This additive property means we can isolate the marginal contribution of a single objective function at a given location.

Similarly, for constrained problems, José Miguel Hernández-Lobato et al. showed in Predictive Entropy Search for Bayesian Optimization with Unknown Constraints that the expected information gain about a constrained optimum splits cleanly into information about the objective plus information about each individual constraint:

$$\text{Total Information Gain} = \text{Gain}_{\text{objective}} + \sum_{i} \text{Gain}_{\text{constraint}_i}$$

This decomposition allows an acquisition function to quantify exactly how much "uncertainty reduction" a cheap constraint evaluation will provide compared to an expensive objective evaluation. The system can then choose the specific task that offers the highest information value per unit of cost.

Balancing the Exploration Space

Decoupling introduces a new challenge: how do we balance exploring the value of the objectives versus exploring the boundaries of the feasible region?

Quoc Phong Nguyen formalizes this in Optimistic Bayesian Optimization with Unknown Constraints by analyzing the trade-off between "vertical exploration" (reducing uncertainty about function values) and "horizontal exploration" (mapping the boundaries of the feasible space). By providing provable theoretical guarantees, the author demonstrates that a decoupled strategy can systematically choose the optimal function to query even under highly asymmetric evaluation costs.

This principle extends beyond Bayesian optimization into broader optimization domains:

  • Large-Scale Continuous Optimization: Nitesh Kumar Singh et al. in Mini-batch stochastic subgradient for functional constrained optimization decouple the sampling of objective functions from the sampling of constraints by using independent mini-batch sizes. This allows practitioners to allocate more computational resources to whichever component is more expensive to evaluate, accelerating overall convergence.
  • Discrete Optimization: In satisfiability problems, Chuan Luo et al. show in Tailoring Local Search for Partial MaxSAT that separating hard and soft constraint scores allows search heuristics to make much more granular, effective decisions based on whether the current priority is achieving feasibility or improving optimality.

Lookahead and Asynchronous Execution

When evaluations can be performed independently, greedy selection strategies can sometimes fall short because they fail to account for how a fast evaluation might change the utility of a subsequent slow evaluation.

To address this, Sam Daulton et al. developed the Hypervolume Knowledge Gradient: A Lookahead Approach for Multi-Objective Bayesian Optimization with Partial Information. By utilizing a one-step lookahead mechanism, their framework explicitly models how querying a specific, potentially low-fidelity or independent objective right now will influence the future optimization trajectory.

This lookahead capability prevents the system from getting stuck in synchronization bottlenecks, enabling highly efficient parallel and asynchronous experimentation where resources are dynamically routed to the most informative metrics.

Key Foundational Work

Bayesian Optimization with Unknown Constraints (2014) — contributed 2 concept elements to this direction.
Efficient top-k processing over query-dependent functions (2008) — contributed 2 concept elements to this direction.
Tailoring Local Search for Partial MaxSAT (2014) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 42 papers
  2. Version 1 · 2026-10-04 · 52 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_decoupled_objective_evaluation_bd656d76,
  title = {decoupled objective evaluation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decoupled-objective-evaluation/rdr_bd656d764a394db482b421d4ef1989e9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_decoupled_objective_evaluation_bd656d76_v2,
  title = {decoupled objective evaluation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decoupled-objective-evaluation/rdr_bd656d764a394db482b421d4ef1989e9/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake