Predictive Entropy Search for Efficient Global Optimization of Black-box Functions

José Miguel Hernández-Lobato, Matthew W. Hoffman, Zoubin Ghahramani

Abstract

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of the expected reduction in the differential entropy of the predictive distribution. This reformulation allows PES to obtain approximations that are both more accurate and efficient than other alternatives such as Entropy Search (ES). Furthermore, PES can easily perform a fully Bayesian treatment of the model hyperparameters while ES cannot. We evaluate PES in both synthetic and real-world applications, including optimization problems in machine learning, finance, biotechnology, and robotics. We show that the increased accuracy of PES leads to significant gains in optimization performance.

Introduction

In many scientific and engineering domains—ranging from robotics to drug discovery—researchers must optimize "black-box" functions. These functions lack a known mathematical form or derivatives and can only be evaluated through expensive, often noisy, experiments or simulations.

The goal of Bayesian Optimization (BO) is to find the global maximum of such functions using a minimal number of evaluations. This is achieved by using an acquisition function to determine the most promising point to sample next. While traditional acquisition functions like Expected Improvement (EI) are popular for their simplicity, they often behave greedily, potentially stalling in local optima or failing to explore the search space sufficiently.

Methodology: The Predictive Entropy Search (PES) Framework

A more principled approach to BO involves maximizing the Information Gain about the location of the global maximum ($x_*$). However, calculating this gain—the core of "Entropy Search" (ES)—is notoriously difficult because the distribution of the global maximum is mathematically intractable. Previous ES implementations relied on complex, heavy approximations that were often inaccurate and struggled to account for hyperparameter uncertainty.

PES overcomes these hurdles through a mathematical "flip." By leveraging the symmetry of mutual information, the authors demonstrate that maximizing the information gain about the global maximum $x_*$ is equivalent to maximizing the information that a new observation $y$ provides about $x_*$: $$\alpha_n(x) = H[p(y|D_n, x)] - \mathbb{E}_{p(x_*|D_n)}[H[p(y|D_n, x, x_*)]]$$

This reformulation offers several advantages:

  1. Analytic Tractability: The first term, the entropy of the predictive distribution, is easily computed for Gaussian Processes.
  2. Efficient Approximation: The second term represents the entropy of the prediction given the location of the maximum. The authors utilize Expectation Propagation (EP) and Bochner’s Theorem (using random Fourier features) to approximate this term efficiently.
  3. Fully Bayesian Treatment: Unlike its predecessors, PES can be integrated with Markov Chain Monte Carlo (MCMC) to marginalize over model hyperparameters, increasing robustness against model mismatch.
Figure 1: Comparison of acquisition function surfaces. PES provides a significantly more accurate approximation of the ground truth (RS Objective) than the original ES method.

Experimental Evaluation

The performance of PES was evaluated against standard benchmarks (EI and ES) across a variety of domains:

  • Synthetic Benchmarks: Standard functions including Branin-Hoo, Cosines, and Hartmann-6.
  • Machine Learning: Tuning hyperparameters for a neural network on the Boston Housing dataset.
  • Biotechnology: Optimizing bacterial hydrogen production conditions.
  • Finance: Optimizing the Sharpe ratio of stock portfolios.
  • Robotics: Tuning bipedal robot controller parameters for maximum walking speed under varying noise levels.

Key Results

In nearly all synthetic and real-world tests, PES achieved lower "immediate regret"—the difference between the true maximum and the algorithm's best guess—compared to ES and EI.

Figure 5: Performance comparison on standard synthetic cost functions. PES (blue) consistently achieves lower median immediate regret across evaluations compared to EI and ES.

Key findings include:

  • Superior Exploration: PES naturally explores more aggressively than EI. By seeking information rather than just immediate improvement, it is less likely to be trapped by local modes in multimodal functions.
  • Robustness to Noise: In high-noise scenarios, such as the robotics tasks, PES significantly outperformed EI, which often failed to explore effectively.
  • Computational Efficiency: While PES requires more computation per iteration than EI, it significantly reduces the number of expensive black-box evaluations required, leading to better overall efficiency in wall-clock time for costly problems.

Conclusion and Future Work

Predictive Entropy Search represents a significant advancement in information-theoretic Bayesian optimization. By shifting the focus to the entropy of predictive distributions, PES provides a more accurate, robust, and efficient path to global optimization. While the current framework is optimized for Gaussian noise and relatively low-dimensional spaces (up to $d=8$), future research will focus on scaling these information-theoretic methods to high-dimensional problems and non-Gaussian likelihoods.

Summary updated

Abstract

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of the expected reduction in the differential entropy of the predictive distribution. This reformulation allows PES to obtain approximations that are both more accurate and efficient than other alternatives such as Entropy Search (ES). Furthermore, PES can easily perform a fully Bayesian treatment of the model hyperparameters while ES cannot. We evaluate PES in both synthetic and real-world applications, including optimization problems in machine learning, finance, biotechnology, and robotics. We show that the increased accuracy of PES leads to significant gains in optimization performance.
BibTeX
@inproceedings{hernandezlobato2014predictive,
  author = {José Miguel Hernández-Lobato and Matthew W. Hoffman and Zoubin Ghahramani},
  title = {{Predictive Entropy Search for Efficient Global Optimization of Black-box Functions}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2014},
  url = {https://openreview.net/forum?id=H1Z0kKW_WS},
}
Figure