Abstract
We develop a new computational approach for "focused" optimal Bayesian experimental design with nonlinear models, with the goal of maximizing expected information gain in targeted subsets of model parameters. Our approach considers uncertainty in the full set of model parameters, but employs a design objective that can exploit learning trade-offs among different parameter subsets. We introduce a new layered multiple importance sampling scheme that provides consistent estimates of expected information gain in this focused setting. This sampling scheme yields significant reductions in estimator bias and variance for a given computational effort, making optimal design more tractable for a wide range of computationally intensive problems.
The Challenge of Focused Experimental Design
In Bayesian Optimal Experimental Design (BOED), the objective is to select experimental conditions $d$ that maximize the Expected Information Gain (EIG). Standard BOED typically seeks to learn all model parameters simultaneously. However, in many scientific contexts, researchers are interested in a specific subset of parameters $\theta$, while other parameters $\eta$ are treated as "nuisance" variables (e.g., sensor noise or calibration offsets).
Calculating EIG for these "focused" designs is computationally demanding because it requires marginalizing out the nuisance parameters within a nested integration framework. Traditional Monte Carlo estimators often suffer from high bias or require prohibitive computational resources to achieve convergence. Furthermore, standard estimators can exhibit a "bias-concentration link," where the estimator's bias is correlated with posterior concentration. This can mislead optimizers into selecting designs that appear to yield high information but are actually artifacts of estimator error.
Layered Multiple Importance Sampling (LMIS)
To address these challenges, we propose Layered Multiple Importance Sampling (LMIS), a two-layer computational scheme designed for efficiency and consistency:
- Layer 1 (The Learning Layer): This layer adaptively learns the posterior landscape by "recycling" samples and model evaluations from previous data realizations. Using a Multiple Importance Sampling (MIS) framework, it combines these samples into a mixture distribution to estimate the moments (mean and covariance) of the current posterior.
- Layer 2 (The Estimation Layer): Based on the moments identified in Layer 1, the algorithm constructs a tailored multivariate $t$-distribution as a biasing distribution. Fresh samples are then drawn to produce an unbiased estimate of the marginal and conditional likelihoods required for the EIG.
Efficiency is further enhanced through mixture pruning—discarding irrelevant samples—and initial sorting of data realizations by prior density to accelerate the "enrichment" of the sample pool.
Experimental Evaluation
The LMIS framework was evaluated across three benchmarks:
- 2D Linear-Gaussian Toy Problem: Visualized the fundamental trade-offs between learning parameters of interest versus nuisance parameters.
- High-Dimensional Linear-Gaussian (4D and 8D): Demonstrated superior convergence rates and significantly lower Mean Squared Error (MSE) compared to standard prior-based sampling.
- Mössbauer Spectroscopy: A real-world nonlinear physics application. LMIS correctly identified the global optimal design for inferring an "isomeric shift," whereas standard estimators failed by converging to a local minimum due to high bias.
Key Insights and Conclusion
The primary innovation of LMIS lies in information sharing. While different experimental outcomes produce different posteriors, these distributions often overlap significantly. By recycling evaluations, LMIS learns the posterior landscape far more efficiently than independent sampling.
Crucially, LMIS is non-intrusive, requiring no gradients or Hessians, making it suitable for "black-box" engineering simulators. As a consistent estimator, it is guaranteed to converge to the true EIG value as sample sizes increase. Future work will focus on integrating LMIS with stochastic optimization algorithms and exploring surrogate model acceleration to further reduce the computational burden in extremely high-dimensional spaces.
Summary updated