Abstract
Bayesian Optimal Experimental Design (BOED) provides a rigorous framework for selecting experiments that maximize information gain. However, calculating the Expected Information Gain (EIG) is computationally intensive, often requiring expensive retraining for each new design. We introduce a design amortization approach that utilizes a novel neural architecture combining set-invariant models and conditional normalizing flows. This method enables a single model to estimate EIG across an infinite design space, achieving a 167x improvement in sample efficiency over standard Nested Monte Carlo methods and significantly reducing the time required for design evaluation.
Introduction
In experimental science and engineering, selecting the most informative design is crucial for efficient resource allocation. The gold standard for this selection is the Expected Information Gain (EIG), which quantifies the reduction in uncertainty regarding a parameter of interest. Despite its utility, EIG is notoriously difficult to compute because it depends on the posterior distribution, which typically lacks a closed-form solution.
Existing approximation methods, such as Nested Monte Carlo (NMC), are computationally prohibitive for large-scale applications. While recent variational approaches offer speed improvements, they lack "amortization"—requiring a new model to be trained from scratch for every unique design. This paper addresses these limitations by introducing Design Amortization, a framework that trains a single neural network to generalize across the entire design space.
Methods
The core of the proposed approach is a neural architecture designed to handle the complexities of experimental data and design contexts simultaneously.
The architecture consists of two primary components:
- Set Invariant Model: Experimental units are often exchangeable. To respect this property, the authors employ a set encoder (using attention mechanisms or Deep Sets) that is invariant to the permutation of experimental units. This allows the model to capture complex interactions between units without being sensitive to their ordering.
- Conditional Normalizing Flows (CNF): To represent high-dimensional, complex posterior distributions, the model uses Normalizing Flows. These flows transform a simple base distribution into a complex one through a series of invertible, differentiable mappings. Crucially, these transformations are conditioned on the design context $d$, allowing the model to adapt its posterior estimate to any given experiment.
Training follows a "cheap-to-expensive" strategy. A model is first trained on a computationally inexpensive lower bound (the Posterior Estimator). This pre-trained model then serves as an importance sampler for a more accurate but expensive Variational Nested Monte Carlo (VNMC) bound, ensuring high-precision EIG estimates with minimal additional simulation cost.
Results and Discussion
The framework was evaluated on a suite of Generalized Linear Models (GLMs), including Normal, Logistic, Binomial, and Multinomial distributions, representing core logic used in real-world experimental analysis.
Key findings include:
- Efficiency: The amortized model was over 3x faster than previous variational methods for initial batches and virtually instantaneous for subsequent design evaluations. It demonstrated a 167x improvement in sample efficiency compared to standard NMC.
- Accuracy: On benchmarks with known ground truth, the amortized VNMC bounds were nearly exact, even as problem complexity increased.
- Architectural Insights: Permutation invariance via attention-based encoders was found to be the most critical factor for performance. Furthermore, the choice of flow transformation (e.g., Affine vs. Rational Quadratic Splines) significantly impacts stability and accuracy depending on the specific model domain.
Conclusion
Design amortization represents a significant step forward for BOED, making the evaluation of large design spaces computationally feasible. By encoding the design context directly into a set-invariant flow-based model, researchers can achieve high-fidelity EIG estimates with unprecedented speed and sample efficiency. Future work will explore the integration of this differentiable evaluator into active optimization loops and its application to high-fidelity, black-box scientific simulators.
Summary updated