Nonmyopic Bayesian Optimization via Batch Proxies and One Shot Planning

This direction is merged into another direction: myopic acquisition functions. Its last version stays readable and citable here.

Version 1 of 18 papers16 conceptsfrozen 2026-10-04

Standard Bayesian Optimization (BO) is inherently "myopic." When selecting the next point to sample, traditional acquisition functions like Expected Improvement only consider the immediate gain of the very next step. This greedy approach is often suboptimal because it fails to account for how a current measurement might better inform future decisions over a fixed experimental budget.

While the "optimal" solution to this problem involves solving a complex dynamic programming problem (the Bellman equation), the computational cost of simulating every possible future branching path is usually prohibitive. Recent research has focused on making "nonmyopic" or lookahead BO practical by using batch approximations and one-shot optimization techniques.

The Batch Proxy: Using Parallelism to Predict the Future

The core intuition behind breaking myopia is that a single sequential step is part of a larger journey. In BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design, Shali Jiang, Henry Chai, and Javier González propose a clever shortcut: instead of simulating a tree of sequential decisions, we can calculate the utility of an optimal batch of future points.

Mathematically, the expected utility of an optimal batch of $k$ points serves as a tight lower bound for the true sequential utility of $k$ steps. By optimizing for a batch and then only executing the first point, the agent inherits the exploratory benefits of a long-term plan without the exponential complexity of a decision tree. This "batch-informed" approach allows the algorithm to "see" where it needs to gather information to benefit future iterations.

One-Shot Optimization of Future Paths

Even with batch approximations, nested optimization—where you optimize the next step while considering the "inner" optimization of future steps—is slow. Shali Jiang, Maximilian Balandat, and Jacob R. Gardner addressed this in Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees.

Their insight was to transform the multi-step lookahead into a "one-shot" problem. Instead of solving a sequence of nested problems, they optimize a single, high-dimensional acquisition function that represents a representative future path. By using the "reparameterization trick" (common in variational inference), they make the entire multi-step tree differentiable. This allows standard gradient-based optimizers to find the best immediate action by considering its impact on a simulated future in a single pass.

Why Shallow Lookahead is Sufficient

A natural question is: how far into the future must we look? In Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead?, Xubo Yue and RK use rollout analysis to show that the law of diminishing returns applies heavily to lookahead.

They found that the most significant performance gains occur when moving from a 1-step (myopic) to a 2-step or 3-step horizon. Beyond that, the accumulation of model errors and the inherent uncertainty of the Gaussian Process mean that deep planning often yields little extra benefit. This justifies the "shallow" horizons used in one-shot and batch-informed methods, proving that we can achieve near-optimal long-term performance with very limited lookahead.

Expanding to Multiple Objectives

The benefits of nonmyopic planning are even more pronounced in complex scenarios like Multi-Objective Bayesian Optimization (MOBO), where the goal is to find a Pareto frontier. Syrine Belakaria and Janardhan Rao Doppa demonstrated in Non-Myopic Multi-Objective Bayesian Optimization that metrics like Hypervolume Improvement (HVI) are particularly well-suited for lookahead.

Because HVI satisfies the Bellman optimality principle, it can be decomposed into a sequence of rewards. By applying batch-informed approximations to MOBO, Belakaria showed that nonmyopic agents are much better at "budgeting" their evaluations to cover the entire Pareto front, whereas myopic agents often get stuck over-refining a single region of the objective space.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_nonmyopic_bayesian_optimization_via_batc_d57b7f5f,
  title = {Nonmyopic Bayesian Optimization via Batch Proxies and One Shot Planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/nonmyopic-bayesian-optimization-via-batch-proxies-and-one-shot-planning/rdr_d57b7f5f53434251a7e0e1899a4da6a8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_nonmyopic_bayesian_optimization_via_batc_d57b7f5f_v1,
  title = {Nonmyopic Bayesian Optimization via Batch Proxies and One Shot Planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/nonmyopic-bayesian-optimization-via-batch-proxies-and-one-shot-planning/rdr_d57b7f5f53434251a7e0e1899a4da6a8/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}