Certified Posterior Failure Probabilities with Boundary-Focused Surrogate Design

Many scientific questions ask for one number from a Bayesian posterior: given sensor data, what is the probability that an uncertain physical system crosses a safety threshold? Examples include the chance that subsurface flow exceeds a critical flux, or that a reactor parameter lies in a hazardous range. The simulator behind the likelihood is too expensive to run inside a sampler, so practitioners replace it with a Gaussian process (GP) emulator, a statistical surrogate that predicts simulator outputs and reports its own uncertainty. They then report the surrogate's answer with no statement of how wrong it might be.

This proposal makes two linked changes. First, we split the error in the reported probability into two parts. One is a likelihood part, measured by the Hellinger distance between the true and surrogate posteriors, restricted to the region near the event. The other is a boundary part: the posterior mass of parameters whose safe-or-unsafe label the surrogate cannot yet determine. Both parts can be estimated from the emulator's uncertainty and from samples already drawn from the surrogate posterior. Second, we choose each new simulator run to shrink this split bound. In effect this is a level-set search, weighted by the posterior and tied to an explicit error certificate.

Core Claim

Existing guarantees for surrogate posteriors are global. Cotter, Dashti and Stuart show that the Hellinger distance between posteriors is bounded by the forward-model error under the prior, and that it controls any posterior expectation. Stuart and Teckentrup turn this into convergence rates for GP emulators built on space-filling designs. Because these bounds are global, they reward accuracy everywhere, which costs a number of design points exponential in dimension.

Boundary-focused methods take the opposite view. Thresholding bandits such as APT and continuous level-set estimators such as TRUVAR and multiscale GP level-set estimation spend samples only where a threshold decision can still flip. However, they ignore any posterior measure entirely.

Our claim is that, for a posterior exceedance probability, the relevant error splits cleanly into a localized Hellinger term and a posterior-weighted misclassification term. Neither term requires global surrogate accuracy. A design rule that targets their sum should reach a given certified accuracy with markedly fewer simulator runs than two kinds of rule: rules that target posterior density alone, such as IP-SUR and the Constraint Set Query (CSQ) of Lartaud, Humbert and Garnier, and rules that target the threshold alone. The advantage should be largest when the threshold sits in the moderate tail of the posterior.

The Bound

We use the following notation:

  • $\mu$ and $\hat\mu$ are the true and surrogate posteriors, with densities $p$ and $\hat p$.
  • $q(\theta)$ is the scalar quantity of interest, for example the outflow flux of the Darcy model.
  • $S=\{q\ge\tau\}$ is the true event, with threshold $\tau$, and $\hat S=\{\hat q\ge\tau\}$ is the surrogate's version of it.

The reported error splits by the triangle inequality, and Cauchy–Schwarz applied to $(\sqrt p-\sqrt{\hat p})(\sqrt p+\sqrt{\hat p})$ on $S$ gives

$$ |\mu(S)-\hat\mu(\hat S)| \;\le\; 2\,d_{H,S}(\mu,\hat\mu)\,\sqrt{\mu(S)+\hat\mu(S)} \;+\; \hat\mu(S\,\triangle\,\hat S). $$

Here $d_{H,S}^2=\tfrac12\int_S(\sqrt p-\sqrt{\hat p})^2$ is the Hellinger distance restricted to $S$, and $S\triangle\hat S$ is the set where the two event definitions disagree.

Setting $f=1_S$ in the expectation inequality of Cotter et al. recovers the first term with the global $d_H$. The localization to $S$ is what allows likelihood accuracy far from the event to be ignored. The square-root factor makes the bound relative for rare events: the probability itself sets how small $d_{H,S}$ must be.

Both terms can be computed as expectations under $\hat\mu$. We write $p/\hat p=\exp(-(\Phi-\hat\Phi))\,\hat Z/Z$, where $\Phi$ is the negative log-likelihood (the data misfit), $\hat\Phi$ is its surrogate, and $Z,\hat Z$ are the normalizing constants (the evidence). The ratio of constants is itself an expectation under $\hat\mu$. Following Lie, Sullivan and Teckentrup, we bound the likelihood error and the evidence error separately, so that a small evidence does not inflate the bound.

High-probability GP envelopes then bound $|\Phi-\hat\Phi|$, and therefore $d_{H,S}$, at each sample. They also give each sample a probability of being misclassified, $\Phi_{\mathcal N}(-|m_q(\theta)-\tau|/\sigma_q(\theta))$, where $\Phi_{\mathcal N}$ is the standard normal cumulative distribution function and $m_q,\sigma_q$ are the GP mean and standard deviation for $q$. Because the event $S$ is unknown, we evaluate the first term on the GP's plausible superset of $S$. This makes the certificate conservative.

Research Program

The method keeps two GP outputs: one for the data misfit, or for the forward observations, and one for $q$. A single simulator call updates both. After each batch of runs, we draw a few thousand samples from $\hat\mu$ by MCMC.

To score a candidate run, we "fantasize" it. The GP posterior variance at every existing sample can be updated in closed form after one hypothetical observation, so the expected drop in both terms of the bound is cheap to compute. We hold the samples fixed and reweight them, rather than rerunning MCMC for every candidate. This is the step that should avoid the cost IP-SUR pays. The resulting acquisition behaves like a posterior-weighted, two-term version of TRUVAR's truncated variance reduction. Points far from the threshold contribute to the boundary term almost nothing, much as an arm with a large gap gets little weight in APT's index. Points with negligible posterior mass contribute to neither term.

On the theory side, we will prove the localized bound under the well-behaved-potential assumptions of Lie et al.. We will also show that the certificate holds with the GP's stated confidence when hyperparameters are known. Two limitations are deliberate:

  • Hyperparameter learning. Bounds that survive learning the GP hyperparameters on the fly are flagged as an open problem for TRUVAR as well. We will measure empirically how often certificates fail when hyperparameters are learned.
  • Dimension. Rates in the number of design points that depend on the dimension of the boundary inside the posterior's support, in the spirit of Shekhar and Javidi, are a stretch goal rather than a promised result.

Baselines

The naive reference is a fixed design. We use a Sobol or Latin hypercube design of the same size, then run MCMC on the frozen surrogate. This is exactly the setting analyzed by Stuart and Teckentrup. It shows what global accuracy buys per simulator call.

Maximum-variance active learning removes the fixed-design objection without using either the posterior or the threshold.

The strongest existing competitor is IP-SUR from Lartaud et al.. It picks runs that minimize expected surrogate error weighted by the current posterior, and the authors prove convergence for it. We also run CSQ from the same paper as its cheaper sibling. For accuracy we found no reported result on Lacuna for IP-SUR or CSQ: the paper's banana, bimodal and neutron-noise comparisons are described qualitatively. The paper does report cost per design iteration: about 1600 seconds for IP-SUR on the 4D neutron problem, about 200 seconds on the 2D banana, about 16 seconds for CSQ, and about 3 seconds for D-optimal design.

The strongest boundary-only competitor is TRUVAR from Bogunovic et al., applied to $q$ over the prior. For its Lake Zurich survey we found no reported numerical F1 value on Lacuna.

Two ablations isolate the key ingredient:

  • Posterior-weighted straddle. This is a single-term heuristic that multiplies a standard level-set score by the estimated posterior density. It tests whether the split bound matters, or whether "weight by posterior" alone is enough.
  • Single-term variants. We keep only the Hellinger term, or only the boundary term, of our acquisition.

All methods get the same simulator budget, the same initial design of size 10D, the same GP kernel, and the same hyperparameter refitting schedule. They also share the MCMC sampler and the number of samples. Wall-clock time is reported separately from the run budget. Our method beats a baseline if it reaches the same true error in the exceedance probability with at least 30% fewer simulator runs, at equal or lower cost per iteration than IP-SUR. Its certificate must also cover the true error in at least the nominal fraction of repeated runs.

Why not just…?

Why not just use IP-SUR or CSQ, since they already target the posterior? Both methods weight design by posterior density, so they refine the interior of the posterior's main region. When the threshold surface passes through the region where most of the posterior mass lies, they may do as well as our method, and our experiments include exactly that configuration so that we do not hide it. When the threshold sits in the posterior's tail, the boundary term in our bound is concentrated where these methods spend little. The tail-placement experiment below decides this. Separately, IP-SUR's reported 1600 seconds per iteration in 4D makes it a poor fit for repeated queries. Our sample-reweighting acquisition can be timed directly against it.

Why not just run a level-set method such as TRUVAR, or a thresholding bandit, on the quantity of interest? These methods map the boundary everywhere the prior allows, including regions the data have ruled out. They also never improve the likelihood surrogate, so the first term of the bound stays unaddressed. On the bimodal benchmark with the threshold crossing near both modes, we expect them to waste runs on boundary segments with low posterior mass and to misstate the probability through likelihood error alone.

Why not just multiply a level-set score by the posterior density? This is the real threat. It is cheap and captures most of the intuition. The two differ when the likelihood surrogate, not the boundary, dominates the error, because the weighted score ignores misfit uncertainty inside the event. The two also differ in that the weighted score yields no certificate. If this ablation matches our method in runs-to-accuracy on every benchmark, we will report the contribution as the certificate plus the principled weighting, not as a better sampler.

Why not just use a large fixed design and trust the emulator? Cvetković et al. show on an advection-diffusion-reaction problem that unaccounted model error yields biased, overconfident posteriors. Stuart and Teckentrup indicate that rates from space-filling designs degrade sharply with dimension. The fixed-design baseline measures this gap directly, and its certificate shows how loose global control is.

Proposed Experiments

Benchmarks. We use three settings. The first is the 2D banana and bimodal posteriors of Lartaud et al.. On these, Hellinger distances and exceedance probabilities can be computed exactly by quadrature on a fine grid. We place thresholds at three positions: through the posterior's main region, in its moderate tail, and through only one of the two modes.

The second is the 2D Darcy flow inverse problem of Dashti and Stuart. We truncate the log-permeability expansion to 4, 8 and 16 modes, take the outflow flux exceeding a critical value as the event, and obtain reference answers from long MCMC runs with the full finite-element solver. The third is the neutron-noise problem, but only if its simulator is obtainable.

Metrics. We report the absolute and relative error of the exceedance probability against the number of simulator runs. On the 2D problems we also report the true Hellinger distance and its restricted version. For the certificate, we report its coverage over 30 random seeds and its tightness, defined as the bound divided by the true error. Wall-clock time per iteration is reported throughout.

Decisive checks. Four results would settle the main questions:

  1. With tail-placed thresholds, our method should need clearly fewer runs than IP-SUR to reach 10% relative error. With thresholds through the posterior's main region, the two methods should be roughly equal. That pattern would confirm the mechanism and not merely a tuning advantage.
  2. The posterior-weighted straddle ablation should fall behind on cases dominated by likelihood error, such as Darcy with few sensors. If it does not, we narrow the claim as stated above.
  3. On the 2D problems, certificates should cover the true error at roughly the nominal rate and be no looser than about ten times the true error. Global Hellinger certificates computed from the fixed design should be much looser.
  4. As the number of Darcy modes grows, the gap between our method and the fixed design should widen, and the gap to IP-SUR should not shrink. This would show that the advantage comes from ignoring irrelevant regions, not from the low-dimensional setting.