Bayesian optimization on amortized search space

Tristan Fauvel, Federico Pavone, Titouan Guillou, Pau Redon Muñoz, Chloé GeoffroyarXiv 2026 · arXiv

TL;DR

The paper introduces BOASS (Bayesian Optimization on Amortized Search Space), a framework that leverages mechanistic simulators to "pre-learn" a low-dimensional search space for Bayesian Optimization. By restricting the search to physically sensible designs and using a geometry-aware "pullback" kernel, BOASS significantly accelerates the discovery of optimal solutions in high-dimensional problems like personalized drug dosing.

What Problem Does This Paper Address?

Bayesian Optimization (BO) is a powerful tool for optimizing expensive "black-box" functions, but it famously suffers from the curse of dimensionality. As the number of parameters grows, the search space expands exponentially, making it nearly impossible to find an optimum within a small budget of experiments.

In many scientific fields, we aren't flying completely blind; we often have mechanistic simulators (e.g., physics engines or biological models) that approximately predict system behavior. However, these simulators are rarely perfect. The challenge is: how can we use a "good-but-imperfect" simulator to guide BO without being led astray by its inaccuracies?

Methods

The authors propose a two-stage approach:

  1. Offline Amortization: They train a neural network to "invert" the simulator. This network learns to map simulator parameters (like a patient's physiology) directly to the "optimal" design (like a drug dosage) predicted by the simulator. The image of this network defines a low-dimensional amortized search space $\mathcal{M}$ consisting only of "sensible" candidates.
  2. Online Bayesian Optimization: Instead of searching the raw, high-dimensional space, BO runs over the inputs to the neural network.
  3. Pullback Kernel: A key innovation is the use of a "pullback" kernel. Instead of measuring similarity in the abstract latent space of the neural network, it measures similarity based on the actual designs produced in the original problem space. This ensures the Gaussian Process surrogate remains "aware" of the problem's physical geometry.
  4. $\pi$BOASS: They also introduce a "soft" variant that uses the simulator-derived information as a prior for the location of the optimum, rather than a hard constraint on the search space.

Datasets

The method was evaluated on two main benchmarks:

  • Synthetic Branin Function: A standard optimization benchmark embedded into higher-dimensional spaces (up to $D=32$) to test how well the method handles ambient dimensionality and simulator bias.
  • Personalized Parkinson’s Dosing: A high-stakes real-world application. The "simulator" is a pharmacokinetic/pharmacodynamic (PK/PD) model of Levodopa-Carbidopa. The goal is to find the optimal 13-dimensional dosing regimen (amounts and timings) for simulated patients with varying physiological parameters.

Key Insights and Ideas

  • Restricting the Search Space is a Power Move: By focusing only on regimens that the simulator deems "reasonable," BOASS avoids wasting experimental budget on nonsensical or dangerous designs.
  • Geometry is Everything: Standard latent-space BO often loses the "feel" of the original problem. The pullback kernel fixes this by ensuring that if two latent points produce similar physical designs, the GP treats them as similar.
  • The Bias-Variance Trade-off: Restricting the search space introduces bias (if the simulator is wrong, the true optimum might be outside $\mathcal{M}$) but drastically reduces variance (the search is much more efficient). The authors prove that for many problems, the efficiency gain far outweighs the bias penalty.
  • Robustness to Misspecification: Surprisingly, BOASS remains competitive even when the simulator is slightly wrong. It only fails when the simulator is "fully misspecified"—i.e., completely disconnected from reality.

Key Results

  • Synthetic Efficiency: In high-dimensional Branin tests, BOASS-k$\Phi$ achieved significantly lower regret than standard BO and state-of-the-art high-dimensional methods like TuRBO and SAASBO, especially as the ambient dimension $D$ increased.
Table 1: BO benchmark for the Branin function optimization. BOASS consistently outperforms baselines in high dimensions (D=16, 32) when bias is low.
  • Clinical Relevance: In the Parkinson's dosing task, BOASS found near-optimal regimens within just 50 iterations, whereas standard BO methods struggled to even find safe dosages.
  • Handling Different Feedback: The authors demonstrated that BOASS works with various types of feedback, including direct concentration measurements and "preference feedback" (where a patient simply says which of two regimens felt better).
Table 2: The four evaluation settings for the Parkinson's dosing use-case, ranging from well-specified to fully misspecified.

Limitations and Future Work

  • Hard Constraints: The "hard" version of BOASS can never find the true optimum if it lies outside the amortized space $\mathcal{M}$. While the $\pi$BOASS variant mitigates this, it's a fundamental trade-off.
  • Simulator Dependency: The method's success relies on having a differentiable simulator (or at least one that can be amortized offline).
  • Theoretical Assumptions: The derived regret bounds rely on assumptions like smoothness and specific GP kernels that might be hard to verify for every real-world simulator.

Future work could involve extending this to evolutionary optimization or multi-arm bandits, and testing the system in actual clinical trials with physician oversight.

Summary updated

Abstract

In real-world black-box optimization problems, a mechanistic simulator is often available that approximately predicts the behavior of the system as a function of the parameters being optimized. Here, we show how an amortized inverse map of such a simulator can be used in Bayesian optimization to restrict the search space while using a Gaussian process kernel that is aware of the original problem geometry. This allows for efficiently leveraging the low intrinsic dimensionality of the optimization problem. We characterize its approximation bias and information gain, and derive an intrinsic-dimension-dependent regret rate under additional geometric and Gaussian Process assumptions. We illustrate the method with an application to drug dosing personalization in Parkinson's disease. We use, as a simulator, a pharmacokinetic/pharmacodynamic (PK/PD) model of a drug, a mechanistic model calibrated on population data with quantified parameter uncertainty. A dosing neural network is trained offline against the model, amortizing the map from patient PK/PD parameters to the regimen predicted to be optimal, and online Bayesian optimization then searches over this learned, low-dimensional representation rather than over raw dosages. Numerical simulations demonstrate the improved performance of our method in this application, as well as under simulator misspecification.
BibTeX
@misc{fauvel2026bayesian,
  author = {Tristan Fauvel and Federico Pavone and Titouan Guillou and Pau Redon Muñoz and Chloé Geoffroy},
  title = {{Bayesian optimization on amortized search space}},
  year = {2026},
  eprint = {2611.26459},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26459},
}
Figure