Abstract
Bayesian optimization (BO) is a popular algorithm for solving challenging optimization tasks. It is designed for problems where the objective function is expensive to evaluate, perhaps not available in exact form, without gradient information and possibly returning noisy values. Different versions of the algorithm vary in the choice of the acquisition function, which recommends the point to query the objective at next. Initially, researchers focused on improvement-based acquisitions, while recently the attention has shifted to more computationally expensive information-theoretical measures. In this paper we present two major contributions to the literature. First, we propose a new improvement-based acquisition function that recommends query points where the improvement is expected to be high with high confidence. The proposed algorithm is evaluated on a large set of benchmark functions from the global optimization literature, where it turns out to perform at least as well as current state-of-the-art acquisition functions, and often better. This suggests that it is a powerful default choice for BO. The novel policy is then compared to widely used global optimization solvers in order to confirm that BO methods reduce the computational costs of the optimization by keeping the number of function evaluations small. The second main contribution represents an application to precision medicine, where the interest lies in the estimation of parameters of a partial differential equations model of the human pulmonary blood circulation system. Once inferred, these parameters can help clinicians in diagnosing a patient with pulmonary hypertension without going through the standard invasive procedure of right heart catheterization, which can lead to side effects and complications (e.g. severe pain, internal bleeding, thrombosis).
1. Introduction
In many scientific and engineering fields—from robotics to drug discovery—optimization tasks often involve objective functions that are computationally expensive or slow to evaluate. Bayesian Optimization (BO) has emerged as a robust framework for these scenarios, utilizing a surrogate model (typically a Gaussian Process) to approximate the objective function and an acquisition function to determine the next sampling point.
While the classic Expected Improvement (EI) acquisition function is widely used, it primarily focuses on the mean predicted improvement, often ignoring the associated uncertainty. This can lead to sampling points with high average improvement but a significant risk of zero actual gain. This paper addresses this gap by introducing Scaled Expected Improvement (ScaledEI), a novel acquisition function designed to favor points offering high improvement with high confidence.
2. Methodology: Scaled Expected Improvement
The core innovation of this work is the derivation of ScaledEI. Unlike standard EI, which only considers the expected value of the improvement random variable, ScaledEI incorporates the variance of this improvement. By dividing the expected improvement by its standard deviation, the function acts as a risk-adjustment mechanism. This scaling pushes the algorithm toward regions where the model is more certain that a significant improvement will occur, rather than gambling on high-variance areas.
Furthermore, the authors address real-world constraints where simulations may fail for certain parameter settings. A secondary Gaussian Process is employed to learn these "failure zones" dynamically, weighting the acquisition function to avoid points likely to cause simulation crashes.
3. Experimental Evaluation
The proposed ScaledEI method was evaluated against a diverse set of challenges, including synthetic benchmarks and a real-world medical application.
3.1 Synthetic Benchmarks
A suite of 12 standard global optimization test functions was utilized to assess performance across varying landscapes. These functions, ranging from 1 to 10 dimensions, are characterized by multiple local minima and complex topographies, as detailed in Table 1.
3.2 Precision Medicine Application
The method was also applied to a complex Partial Differential Equation (PDE) model of the human pulmonary blood circulation system. This application aims to estimate physiological parameters, such as vessel stiffness, to assist in the non-invasive diagnosis of pulmonary hypertension using MRI data.
4. Key Results and Discussion
Experimental results demonstrate that ScaledEI is a highly competitive acquisition function. In head-to-head comparisons with state-of-the-art methods like Max-Value Entropy Search (MES) and standard EI, ScaledEI performed significantly better or equal in 67-92% of the test cases.
When compared to traditional global solvers like Genetic Algorithms, ScaledEI reached the global optimum using approximately 100 times fewer function evaluations, highlighting its efficiency for expensive simulations. In the medical application, ScaledEI successfully estimated critical physiological parameters within just 100-200 iterations. This suggests a path toward diagnosing patients in under an hour using non-invasive MRI data, potentially replacing risky invasive procedures like heart catheterization.
5. Conclusion and Future Work
ScaledEI offers a robust and efficient alternative for Bayesian Optimization, performing consistently well across various mathematical landscapes without the need for manual tuning. The authors suggest it as a powerful new "default" choice for researchers using BO. Future work will explore extending ScaledEI to multi-objective optimization and refining constraint modeling through more sophisticated classification techniques. Additionally, validating the algorithm on real-world clinical datasets remains a priority for advancing non-invasive medical diagnostics.
Summary updated