Abstract
Bayesian optimal experimental design (BOED) provides a powerful, decision-theoretic framework for selecting experiments so as to maximise the expected utility of the data to be collected. In practice, however, its applicability can be limited by the difficulty of optimising the chosen utility. The expected information gain (EIG), for example, is often high-dimensional and strongly non-convex. This challenge is particularly acute in the batch setting, where multiple experiments are to be designed simultaneously. In this paper, we introduce a new approach to batch EIG-based BOED via a probabilistic lifting of the original optimisation problem to the space of probability measures. In particular, we propose to optimise an entropic regularisation of the expected utility over the space of design measures. Under mild conditions, we show that this objective admits a unique minimiser, which can be explicitly characterised in the form of a Gibbs distribution. The resulting design law can be used directly as a randomised batch-design policy, or as a computational relaxation from which a deterministic batch is extracted. To obtain scalable approximations when the batch size is large, we then consider two tractable restrictions of the full batch distribution: a mean-field family, and an i.i.d. product family. For the i.i.d. objective, and formally for its mean-field extension, we derive the corresponding Wasserstein gradient flow, characterise its long-time behaviour, and obtain particle-based algorithms via space-time discretisations. We also introduce doubly stochastic variants that combine interacting particle updates with Monte Carlo estimators of the EIG gradient. Finally, we illustrate the performance of the proposed methods in several numerical experiments, demonstrating their ability to explore multimodal optimisation landscapes and obtain high-utility batches in challenging examples.
1. Introduction
Bayesian Optimal Experimental Design (BOED) is a decision-theoretic framework for selecting experiments that maximize the Expected Information Gain (EIG). However, optimizing EIG is notoriously difficult due to its high-dimensionality and non-convexity, especially in batch settings where multiple experiments are designed at once. Standard gradient methods often fail by getting trapped in local optima, failing to explore the full design space effectively.
2. Methodology
The authors address these challenges through a "probabilistic lifting" strategy. Instead of searching for a single optimal point $\xi^*$, they search for an optimal probability distribution $\mu^*$ over the design space.
2.1 Entropic Regularization
By defining a "free energy" functional that combines the EIG with an entropic regularizer (a KL-divergence term), the optimization problem becomes strictly convex in the space of measures. This ensures a unique global optimum in the form of a Gibbs distribution.
2.2 Wasserstein Gradient Flows (WGF)
To find this distribution, the authors use WGF—essentially performing steepest descent on the landscape of probability distributions. This framework naturally incorporates a diffusion term that allows the optimization to escape local basins of attraction and discover the global maximum.
2.3 Scalable Particle Systems
To handle large batches, the authors introduce two tractable restrictions: a mean-field family and an i.i.d. product family. These are implemented using Doubly Stochastic Interacting Particle Systems (IPS), combining particle interactions with Monte Carlo estimation of the EIG gradient.
3. Experimental Evaluation
The methods were evaluated on several benchmarks, ranging from synthetic toy problems to established real-world models:
- Multimodal Landscapes: In 1D and 2D tests, WGF consistently found the global optimum regardless of initialization, whereas standard Gradient Ascent (GA) failed in nearly 50% of runs.
- Batch Design on the Torus: The i.i.d. and repulsive WGF variants significantly outperformed joint optimization for large batches (up to $m=1000$).
- Pharmacokinetic (PK) and FitzHugh–Nagumo (FHN) Benchmarks: The WGF methods were competitive with and often outperformed state-of-the-art specialized BOED baselines like Coordinate Exchange (CE) and Annealed SMC.
4. Discussion and Conclusion
By lifting the BOED problem into the space of measures, this work transforms a non-convex point optimization into a convex measure optimization. This shift enables robust global search and scalable batch design. Future work will address adaptive hyperparameter tuning for the regularization strength and more efficient handling of design constraints.
Summary updated