Robust Bayesian Experimental Designs in Normal Linear Models

Anirban Dasgupta, W. J. StuddenThe Annals of Statistics 1991 · DOI

Introduction

In experimental design, the objective is to select inputs—such as dosage levels or sensor placements—that yield the most informative data. While traditional frequentist methods focus on long-run performance (e.g., A- or D-optimality), standard Bayesian approaches rely on a single prior distribution to represent existing knowledge. However, Bayesian designs can be fragile; if the prior is misspecified, the "optimal" design may perform poorly. This paper bridges this gap by developing "robust" Bayesian designs that maintain high performance even when prior beliefs are slightly inaccurate.

Modeling Prior Uncertainty

The authors address the challenge of finding a design that minimizes Bayes risk for a favored prior while remaining safe if the true prior belongs to a broader family of distributions. They focus on Normal Linear Models with Gaussian, homoscedastic errors and analyze two primary ways to model uncertainty in the prior distribution ($\theta$):

  1. The $\Gamma_1$ Family (Conjugate Priors): In this model, the prior mean is fixed, but the variance-covariance matrix is uncertain, bounded between two precision levels ($lI$ and $kI$). While mathematically tractable, this approach is relatively restrictive.
  2. The $\Gamma_2$ Family (Density Bands): This more flexible "metric neighborhood" bounds the prior density between two functions ($L$ and $U$), allowing both the mean and variance to vary.

To identify the optimal design, the researchers optimize several "robustness functionals," including the maximum regret across the family of priors, the range of possible Bayes risks, and the diameter of the set of all possible Bayes estimates.

Theoretical Benchmarks and Insights

As a theoretical statistics work, the methods are evaluated using classic experimental design benchmarks rather than empirical datasets. These include simple linear regression, quadratic regression, and completely randomized designs.

The "Universal" Design

A striking finding of the research is that for the $\Gamma_2$ (density band) family, a single design is often optimal across a wide variety of problems. Whether the goal is parameter estimation, hypothesis testing, or constructing confidence intervals, the robust design remains consistent. This "universality" simplifies the design process significantly.

Mathematical Properties

The authors demonstrate that their robustness functionals are "decreasing and convex" on the space of information matrices. This property ensures that standard optimization tools can reliably find the global optimum. Furthermore, for the $\Gamma_2$ family, the design minimizing the volume of a Bayesian confidence set is equivalent to a "Bayesian D-optimal" design relative to the base prior $L$.

Key Results

One of the paper's central contributions is Theorem 4.1, which proves that for linear combinations of coefficients ($c'\theta$), several distinct criteria—minimizing the range of the posterior mean, the size of confidence sets, the risk of a hypothesis test, and the range of posterior probabilities—are mathematically equivalent to minimizing a single quadratic form.

In practical terms, the authors illustrate quantitative trade-offs. For example, in a randomized design with two treatments, sacrificing only 2% of the "subjective" Bayes risk (the risk if the favored prior is perfect) can yield a 14% gain in robustness against prior misspecification. In quadratic regression models, robust designs typically involve sampling at the endpoints (0 and $\pm 1$), with proportions determined by the prior variances of the intercept and slope.

Conclusion and Future Directions

The results presented are particularly applicable to group decision-making and the reconciliation of expert opinions with differing priors. However, the study has limitations: it assumes a known error variance ($\sigma^2$) and focuses exclusively on normal linear models. Future research could extend these robust designs to hierarchical Bayesian models where variance is unknown, or to non-linear models such as logistic regression.

Summary updated

Abstract

We address the problem of finding a design that minimizes the Bayes risk with respect to a fixed prior subject to being robust with respect to misspecification of the prior. Uncertainty in the prior is formulated in terms of having a family of priors instead of one single prior. Two different classes of priors are considered: $\Gamma_1$ is a family of conjugate priors, and a second family of priors $\Gamma_2$ is induced by a metric on the space of nonnegative measures. The family $\Gamma_1$ has earlier been suggested by Leamer and Polasek, while $\Gamma_2$ was considered by DeRobertis and Hartigan and Berger. The setup assumed is that of a canonical normal linear model with independent homoscedastic errors. Optimal robust designs are considered for the problem of estimating the vector of regression coefficients or a linear combination of the regression coefficients and also for testing and set estimation problems. Concrete examples are given for polynomial regression and completely randomized designs. A very surprising finding is that for $\Gamma_2$, the same design is optimal for a variety of different problems with different loss structures. In general, the results for $\Gamma_2$ are significantly more substantive. Our results are applicable to group decision making and reconciliation of opinions among experts with different priors.
BibTeX
@article{dasgupta1991robust,
  author = {Anirban Dasgupta and W. J. Studden},
  title = {{Robust Bayesian Experimental Designs in Normal Linear Models}},
  journal = {The Annals of Statistics},
  year = {1991},
  doi = {10.1214/aos/1176348247},
}
Figure