Grading the Synthetic Prior Ensemble Before Running the Inversion

Ensemble Kalman inversion (EKI) recovers unknown physical fields, such as the permeability of rock underground, from a few noisy indirect measurements. It does this by moving a small cloud of candidate fields, called particles, without ever computing derivatives. Those starting particles are almost always synthetic: they are draws from a mathematical prior such as a Gaussian random field, or from a procedural generator. EKI also has a well-known rigidity, the subspace property. Every update is a linear combination of the current particles, so the final answer can never leave the linear span of the initial synthetic draws. In EKI, then, depending on synthetic data is a geometric fact and not only a modeling worry. Whatever the synthetic ensemble fails to span, the inversion cannot recover.

We propose to treat the initial ensemble as a synthetic dataset and to evaluate it with the tools of synthetic-data evaluation, before and during inversion. The note makes two concrete contributions. The first is a coverage gap, computed from the generator alone, that lower-bounds the reachable error. The second is a span residual, computed from the observations using only the forward runs EKI already performs, that flags a misspecified synthetic prior on real data where the true field is unknown. We ask whether these scores work as a "compass" (they rank priors correctly) or also as a "ruler" (they predict absolute error). We then use the span residual to decide when and from which generator family to add particles. The bet is that much of EKI's failure on realistic problems comes from bad synthetic coverage rather than from the optimizer. If so, cheap diagnostics of coverage should matter more than further ways of breaking the subspace.

Core Claim

Analysis of the Ensemble Kalman Filter for Inverse Problems shows that EKI's continuous-time limit is a preconditioned gradient flow whose initial span is invariant. Convergence Analysis of Ensemble Kalman Inversion: The Linear, Noisy Case shows that this confinement acts as implicit regularization and that stopping by Morozov's discrepancy principle is needed under noise. Together they imply a clean decomposition.

Let $m_0$ be the initial ensemble mean and $S$ the span of the initial deviations. The EKI estimate $\hat u$ then lies in $m_0 + S$. For any true field $u^\dagger$ and any norm $\|\cdot\|_C$ weighted by a reference covariance $C$, the error is bounded below by the part of the truth the ensemble cannot represent:

$$\|\hat u - u^\dagger\|_C \;\ge\; \|P_S^{\perp}(u^\dagger - m_0)\|_C =: \mathrm{gap}(S, u^\dagger),$$

where $P_S^{\perp}$ is the $C$-orthogonal projection off $S$. The gap needs no forward solves. That makes it a property of the synthetic generator that can be measured on held-out reference fields.

On real data the truth is unknown, so we pair the gap with a data-side score. Let $\Delta G$ be the matrix of centered forward outputs of the initial particles, $\Gamma$ the noise covariance, and $y$ the observations. The span residual is

$$r(S) = \min_{a}\, \big\|\Gamma^{-1/2}\big(y - \overline{G} - \Delta G\, a\big)\big\|,$$

where $\overline{G}$ is the mean forward output and $a$ is a vector of ensemble weights. It asks whether any member of the linearized span can fit the data down to the noise level. If $r(S)$ stays well above $\sqrt{\dim y}$, the level Morozov's principle stops at, the synthetic prior lacks a mode that the data demands. Both scores cost nothing beyond what EKI already spends.

The claim we test has two parts. First, in linear and mildly nonlinear problems, the expected gap of a generator is a ruler: it predicts final EKI error after discrepancy stopping across generator families. Second, the span residual is at least a compass: it ranks candidate priors on data without ground truth. A further question follows from the second part. If adding particles from an alternative generator only when $r(S)$ fires matches or beats generic subspace-breaking methods at equal forward-run budget, then targeted coverage repair is a better use of compute than random escape.

Relation to Prior Work

Three existing lines of work loosen the subspace constraint, and each motivates this note while differing from it.

Dropout Ensemble Kalman Inversion for High Dimensional Inverse Problems randomly zeroes coordinates of particle deviations. With this it proves exponential convergence with $J \ll d_u$ under strong convexity. It was tested only where the simulated truth matches the prior and solver. Our question is what happens when the synthetic prior is wrong, and whether coordinate-aligned escape finds non-axis-aligned structure such as channels.

Parameterizations for ensemble Kalman inversion escapes the span by updating hyperparameters of a non-centered prior, or by using level sets. The transformation family must be fixed in advance. We use its channelized and inclusion fields as "reality" generators, and its hierarchical EKI as a competitor.

Ensemble Kalman methods with constraints solves updates as quadratic programs inside the range of the covariance. It reports that "up to 30% of particles violated biological constraints" without this, and that 100% stayed plausible with it. It constrains our design: any particles we add must pass through the same projection.

On the synthetic-data side, two papers supply the evaluation method. Efficacy of Synthetic Data as a Benchmark separates ranking agreement, measured by Spearman rank correlation (SRCC), from absolute agreement, measured by mean squared performance difference (MSPD). It finds synthetic benchmarks reliable as compasses on shallow tasks (SRCC 0.94 on SNIPS intent detection) but not on structured ones (SRCC as low as 0.20 for political NER). On the Equivalency, Substitutability, and Flexibility of Synthetic Data defines equivalency ratios and substitutability, reporting about 30:1 for image frames in tracking. We import both definitions into a setting where no one has measured them: the composition of the initial ensemble. Is this Model for Real? motivates the real-data logic. It judges a model by whether real-data fits differ from fits on simulated data, and our span residual is a cheap analogue of that test.

Research Program

We first establish when the gap bound is close to tight. For linear forward maps, EKI with discrepancy stopping converges to a regularized fit within $m_0 + S$. We will derive how far the achieved error exceeds the gap as a function of noise level and the conditioning of the forward map restricted to $S$. This tells us when the gap is a ruler and when it is only a compass. For nonlinear maps such as Darcy flow, the span is still invariant, but the linearized residual is only approximate, so the question becomes empirical.

Second, we build a generator zoo. It contains Gaussian random fields with several Matérn length-scales, Cauchy-type heavy-tailed fields, level-set channel and inclusion generators in the style of Chada et al., and a mixture. Every generator in the zoo serves in turn as prior and as held-out "reality," so we get a full mismatch matrix instead of one favourable pairing.

Third, we measure equivalency and substitutability for ensembles. The question is how many Matérn draws buy the same final error as one draw from the true generator family, and what fraction of a well-matched ensemble can be swapped for mismatched draws before error rises. The curse of large ensembles proven by Schillings and Stuart predicts that these curves are non-monotone under noise, which would make them unlike their perception-domain counterparts.

Fourth, we test the repair rule. It runs the initial $J$ forward evaluations and computes $r(S)$. If $r(S)$ exceeds the noise level by a set margin, it adds $k$ particles drawn from the zoo family whose samples most reduce the linearized residual, projects them with the constraint quadratic program, and continues EKI. Scoring candidates from each family costs forward runs, and those runs are charged to the budget.

Baselines

The naive reference is standard EKI initialized from a single default Matérn prior with discrepancy stopping, at ensemble size $J$. This is what practitioners do, and it is the setting in which the gap bound bites hardest.

The strongest competitor for recovering structure outside a misspecified span is Dropout EKI. It is the only method in the file with proven small-ensemble convergence and linear scaling in dimension. A second strong competitor is hierarchical or level-set EKI, which adapts the prior instead of escaping it.

For fairness, every method gets the same total number of forward-model evaluations, the same observation locations and noise realizations, the same stopping rule, and the same tuning budget of twenty configurations each. Tuned settings include the dropout rate for Dropout EKI and the hyperprior ranges for hierarchical EKI. Constraint-respecting variants of all methods use the quadratic program projection. We could find no reported numerical Darcy-flow error for Dropout EKI or for hierarchical EKI on Lacuna, so both are rerun here.

Beating them means lower median relative field error and lower Fourier-spectrum error ($f$RMSE, a metric from PDEBench) under mismatched priors, at equal budget, across at least three of the four mismatch types, without losing more than a small margin when the prior is correctly specified.

The key ablation replaces the residual-triggered, family-selected augmentation with the same number of extra particles drawn either from the original prior or as isotropic Gaussian perturbations. This isolates whether the gain comes from choosing the right synthetic family or merely from having more particles.

Why not just…?

Why not just increase the ensemble size until it spans the space? When $d_u$ runs into the tens of thousands, $J \ge d_u$ forward solves per iteration is infeasible. More importantly, Schillings and Stuart prove that under noise a larger $J$ can raise parameter error while misfit falls. Our equal-budget comparison includes a "spend the augmentation budget on more prior draws" arm, which settles this alternative directly.

Why not just use Dropout EKI, which already breaks the subspace? This is the real threat. If dropout recovers channels under a Matérn prior as well as targeted augmentation does, our method reduces to a diagnostic. Its analysis assumes strong convexity and a matched prior, and coordinate masking has no reason to favour geometrically coherent directions. The channel- and inclusion-truth cells of the mismatch matrix are where this is decided. We will report them even if dropout wins, and we will also test the span residual as a trigger for dropout.

Why not just adopt a hierarchical prior and let it adapt? The hierarchical prior only adapts within its chosen transformation family. If reality lies outside that family, for example channels under a length-scale hierarchy, it inherits the same gap. We will measure this as a gap computed on the hierarchical prior's reachable set.

Why not just check the error after inversion? On real data there is no truth to check against. The span residual exists precisely because a pre-inversion, truth-free alarm is what practitioners lack.

Proposed Experiments

The main testbed is 2D Darcy flow with 49 noisy pressure observations, following the setting of Schillings and Stuart. Truth fields come from every zoo generator and from the coefficient fields in PDEBench (code and data), used as a distribution independent of all our priors. A forward-model-mismatch variant generates data at PDEBench resolution and inverts with a coarser solver. The 1D elliptic and linear transport problems serve as testbeds where the gap bound can be checked exactly. Metrics are relative field error, $f$RMSE, the fraction of constraint-violating particles, and forward-run counts.

The first decisive check is the ruler and compass test. Across the mismatch matrix, we compute SRCC and MSPD between the expected gap and the achieved EKI error, and between $r(S)$ and achieved error. If SRCC for $r(S)$ falls below roughly 0.7 on nonlinear Darcy, the truth-free alarm fails, and the note reduces to the linear theory.

The second check measures equivalency and substitutability curves for ensemble composition, including whether they are non-monotone in $J$ under noise.

The third check is the equal-budget comparison against Dropout EKI and hierarchical EKI, together with the augmentation ablation.

Finally, we run a real-data plausibility check on the PhysioNet type 2 diabetes series used by Albers et al. Candidate synthetic patient-parameter priors are ranked by $r(S)$. We then test whether that ranking matches held-out glucose prediction error, which asks whether the compass survives contact with real data.