Differentiable Acquisition Functions for Bayesian Optimization

29 concepts23 papers1 proposal

Bayesian Optimization (BO) is a powerful strategy for optimizing expensive black-box functions. It works by maintaining a probabilistic surrogate model of the objective and using an acquisition function to decide where to sample next. However, as Hutter et al. point out, the "inner loop" of BO—the process of maximizing this acquisition function—is often treated as a secondary black-box problem.

Historically, practitioners relied on gradient-free search methods like random search or genetic algorithms to find the next query point. These methods scale poorly as the dimensionality of the search space increases. The research direction described here shifts this paradigm by making acquisition functions differentiable, allowing us to use efficient first- and second-order optimization methods to select the best experiments.

The Reparameterization Trick for Acquisition

The primary challenge in differentiating acquisition functions is that they often involve expectations over a posterior distribution. For example, the Expected Improvement (EI) is the average improvement we expect to see over the current best value. When these expectations lack a closed-form solution, we typically estimate them using Monte Carlo (MC) sampling.

To compute gradients through these samples, researchers apply the reparameterization trick. As explained by Balandat et al. and Hutter et al., this involves expressing the random variables (the posterior samples) as a deterministic transformation of a fixed base distribution (like a standard Normal). By doing so, the randomness is "pushed" outside the gradient operator, making the acquisition function fully differentiable with respect to the input candidates. This enables the use of auto-differentiation and high-order optimizers like L-BFGS or Adam, which Balandat et al. show can be an order of magnitude faster than gradient-free alternatives.

Scaling to Parallel and Multi-Objective Settings

Differentiability becomes even more critical when we move beyond simple, single-point optimization. In many real-world scenarios, we want to evaluate a batch of points in parallel or optimize multiple conflicting objectives.

  • Batch Optimization: Selecting a batch of $q$ points requires searching a $q \times d$ dimensional space. Marmin et al. derived exact analytical gradients for the multipoint Expected Improvement ($q$-EI) by applying Plackett’s formula to differentiate multivariate normal distributions. Similarly, Teufel et al. developed BEEBO, a batch acquisition function that uses differentiable approximations from statistical physics to avoid the computational overhead of Monte Carlo sampling entirely.
  • Multi-Objective Optimization: In multi-objective BO, we seek the Pareto frontier. Balandat et al. introduced $q$EHVI, a differentiable version of the Expected Hypervolume Improvement. By making the hypervolume calculation compatible with auto-differentiation, the authors enabled efficient parallel candidate selection for complex engineering trade-offs. Deng et al. further simplified this by showing that EHVI is mathematically equivalent to a standard hypervolume improvement in a transformed space, providing exact analytic gradients that accelerate the search by orders of magnitude.

Non-Myopic Planning and the Envelope Theorem

Standard acquisition functions are "myopic"—they only consider the immediate benefit of the next sample. "Non-myopic" or lookahead methods consider how a sample today will improve our knowledge for future decisions. These methods are notoriously difficult to optimize because they involve nested optimizations (e.g., "what is the best point to pick after I see the result of this point?").

To solve this, Wu et al. and Wu and Frazier utilize the envelope theorem. This theorem allows one to compute the gradient of the value of an optimization problem without needing to differentiate through the optimizer itself. By combining this with Sample Average Approximation (SAA), the authors transformed slow, nested lookahead calculations into one-shot deterministic optimization problems that can be solved in seconds.

Variational Bounds for Information Gain

Information-theoretic acquisition functions, such as Entropy Search, are powerful but computationally expensive because they require estimating the Mutual Information between the sample and the global optimum.

Foster et al. addressed this by framing Bayesian Optimal Experimental Design (BOED) as a unified differentiable problem. They introduced variational lower bounds that allow for the simultaneous estimation of information gain and the optimization of experimental designs via stochastic gradient ascent. This approach was extended to multi-objective settings by Qing et al., who used differentiable approximations of output space partitioning to enable gradient-based batch selection for finding Pareto frontiers.

By treating the acquisition process as a differentiable program, this body of work has turned the "inner loop" of Bayesian Optimization from a computational bottleneck into a scalable, high-performance engine for discovery.

Key Foundational Work

Differentiating the Multipoint Expected Improvement for Optimal Batch Design (2015) — contributed 1 concept elements to this direction.
Bayesian Optimization with Gradients (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_differentiable_acquisition_functions_e9fc6db8,
  title = {differentiable acquisition functions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/differentiable-acquisition-functions/rdr_e9fc6db8237f4d1d85e7cf8af0fca945},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_differentiable_acquisition_functions_e9fc6db8_v1,
  title = {differentiable acquisition functions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/differentiable-acquisition-functions/rdr_e9fc6db8237f4d1d85e7cf8af0fca945/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake