A Bayesian-Geometric Reading of Classical Model-Based Optimal Design

Mohamed TarekarXiv 2026 · arXiv

TL;DR

This paper provides a unified "Bayesian-geometric" bridge to classical frequentist experimental design. By using the Bernstein–von Mises theorem, the author demonstrates that famous criteria like D-, A-, and T-optimality are actually specific asymptotic limits of the expected "distance" (dissimilarity) between a prior and a posterior distribution.

What Problem Does This Paper Address?

In the world of Optimal Experimental Design (OED), there has long been a divide between two schools of thought.

The classical frequentist approach uses "alphabetical" criteria (D, T, A, L, C, E) based on the Fisher Information Matrix (FIM). These are motivated by the Cramér–Rao bound, aiming to minimize the variance of future estimators. The Bayesian approach, on the other hand, focuses on maximizing an "expected utility," such as the Expected Information Gain (EIG), which measures how much an experiment changes our beliefs.

While it was already known that D-optimality relates to the Bayesian EIG, the other classical criteria often felt like a disconnected collection of heuristics. This paper addresses this gap by providing a single mathematical template that recovers all five major classical criteria from Bayesian principles.

Methods

The author’s core technique involves looking at the Bernstein–von Mises (BvM) limit. This theorem states that as the number of observations ($N$) grows to infinity, the posterior distribution becomes Gaussian, centered at the Maximum Likelihood Estimate, with a covariance determined by the FIM.

The paper defines a "Bayesian-geometric template" where an optimal design is chosen to maximize the expected dissimilarity between the prior and the posterior. By varying three ingredients in this template, the author derives the classical criteria:

  1. Dissimilarity Measure: Using forward Kullback–Leibler (KL) divergence, reverse KL divergence, or the 2-Wasserstein distance.
  2. Prior Parametrization: Specifically, how the weight matrices in classical criteria map to the covariance of a Gaussian prior.
  3. Approximation Scheme: Using Taylor expansions (specifically for the matrix square root in the Wasserstein case) to find the "leading-order" term as $N \to \infty$.

Datasets

Not discussed in the paper. This is a theoretical and mathematical derivation paper; it does not use empirical datasets or simulations for evaluation.

Key Insights and Ideas

  • The Unified Template: The most significant insight is that D, T, A, L, and C-optimality aren't just arbitrary functions of the FIM; they are different ways of measuring the "geometric separation" between what we knew before the experiment and what we know after.
  • KL Divergence Asymmetry: The paper shows that the choice of "direction" in KL divergence matters. Forward KL (Posterior relative to Prior) leads to D-optimality (volume-based), while reverse KL (Prior relative to Posterior) leads to T-optimality (trace-based).
  • Wasserstein and A-Optimality: The author provides a novel derivation of A-, L-, and C-optimality as first-order approximations of the squared 2-Wasserstein distance. This links transport-based distances directly to classical variance-minimization.
  • E-Optimality as Robustness: E-optimality (maximizing the minimum eigenvalue) is shown not to be a direct dissimilarity limit, but rather a "min-max" robust version of C-optimality. It represents a designer who wants to do well even if the modeler's prior direction is unknown or untrusted.
  • Designer vs. Modeler: The paper clarifies the distinction between the modeler’s prior (used for inference) and the designer’s prior (used to average over possible truths at design time), showing how they interact to recover the classical EIG.

Key Results

The paper successfully derives the following leading-order limits as $N \to \infty$:

  • D-optimality: Recovered from $E[KL(Posterior \parallel Prior)]$.
  • T-optimality: Recovered from $E[KL(Prior \parallel Posterior)]$.
  • L-, C-, and A-optimality: Recovered from the expected approximate squared 2-Wasserstein distance.
  • E-optimality: Recovered as robust C-optimality against uncertainty in the prior direction.

The author notes that the derivations for T-, L-, C-, and A-optimality from these specific dissimilarities appear to be novel contributions to the literature.

Limitations and Future Work

  • Asymptotic Reliance: The derivations rely on the large-sample limit ($N \to \infty$). At small or moderate sample sizes, the specific shape of the prior and non-Gaussianity of the posterior become much more important, and these classical approximations may be less accurate.
  • Computational Complexity: Evaluating these Bayesian objectives in practice often requires nested Monte Carlo or variational approximations, which are significantly more expensive than simply computing the FIM.
  • Model Assumptions: The work assumes the model is correctly specified and the parameters are identifiable. Extending this to misspecified or non-identifiable models would require modified BvM limits.
  • The $\Phi_p$ Family: While this paper unifies the criteria through different distances, it remains an open question if other members of Kiefer’s $\Phi_p$ family of criteria can be mapped to other natural dissimilarity functions.

Summary updated

Abstract

The classical model-based optimal design criteria (D-, T-, L-, C-, and A-optimality) are usually motivated via the Cramér--Rao bound as optimizing scalar functions of the Fisher information matrix (FIM). This paper offers a complementary reading in which each criterion appears asymptotically as $N \to \infty$ as the leading-order term of an expected dissimilarity between prior and posterior, with the FIM entering through the Bernstein--von Mises limit. The Kullback--Leibler (KL) divergence of the posterior relative to the prior recovers D-optimality (classically known); a reverse KL of the prior relative to the posterior recovers T-optimality (here defined as maximization of the trace of the FIM, following one common convention) and its weighted-trace generalization; and a first-order approximation of the squared 2-Wasserstein distance recovers L-, C- and A-optimality. The five criteria emerge as instances of a common Bayesian-geometric template, indexed by three choices: the dissimilarity, the prior parametrization, and the approximation scheme used to evaluate it (e.g.\ Taylor expansions of the matrix square root). For the weighted criteria, the Gaussian prior covariance encodes the criterion's weight matrix, so any choice of weights corresponds to a specific prior in the family. The T-, L-, C-, and A-optimality derivations appear novel to the best of my knowledge. A sixth criterion, E-optimality, is recovered not as an independent expected-dissimilarity limit but as a min-max robustification of C-optimality against uncertainty in the modeler's prior direction. The framework is extended to distinguish between a designer's prior over unknown parameters at design time and a modeler's prior used for posterior inference, recovering the classical expected information gain objective as a special case.
BibTeX
@misc{tarek2026bayesian,
  author = {Mohamed Tarek},
  title = {{A Bayesian-Geometric Reading of Classical Model-Based Optimal Design}},
  year = {2026},
  eprint = {2611.04450},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04450},
}
Figure