Training Neural Emulators on the States a Filter Actually Visits

Fast neural emulators (networks trained to imitate a physics solver one time step at a time) are increasingly used as the forecast model inside data assimilation. Data assimilation is the loop that alternates between running a forecast and correcting it with noisy, sparse sensor readings. In that loop the emulator never sees the clean simulator states it was trained on. It sees corrected states, built by nudging an ensemble of guesses toward the observations. Those corrections are statistically odd: they are smooth where sensors are dense, rough where they are absent, and seamed at the edges of the regions used for local updates. The emulator then extrapolates from inputs it has never seen. Its errors grow over the steps between observations. The forecast ensemble lands farther from the next observation. In a high-dimensional field, a larger forecast–observation gap is exactly what drives particle filters into weight collapse, where one guess takes all the weight. We think this feedback loop, more than any of its three parts alone, is why surrogate-based filters degrade in large systems.

The proposal is simple to state. Run the filter during training. Collect the corrected states it produces. Label each one with what the true solver would do from that state, and fine-tune the emulator on those pairs, repeating as the emulator improves. This is the logic of imitation learning with an expert, where the learner is trained on the states its own policy visits. Here it is applied to the forecast–analysis cycle of a spatially localized ensemble filter. We expect three effects at a fixed ensemble size: higher effective sample size in each local region, longer usable gaps between observations, and slower decay of free forecasts launched from filter analyses.

Core Claim

Each of the three research directions contains part of the problem. Autoregressive error accumulation is the drift that comes from feeding a model's own outputs back as inputs. The usual explanation is the mismatch between training on true inputs and running on predicted ones, as described by Peng et al.. Prediction horizon decay is the loss of skill, sharpness and calibration as lead time grows. The curse of dimensionality in particle filtering means that importance weights degenerate unless the number of particles grows rapidly with the effective dimension of the update (Albarakati et al.). That paper also reports that shrinking the observation dimension was the main lever against degeneracy.

Our claim connects these. When the forecast model is an emulator, the distance between forecast particles and the posterior is not set by chaos alone. It is inflated by emulator error on off-distribution analysis states, and that inflated distance is what the weights "see." If the emulator is made accurate on the specific distribution of analysis states, both the filter's degeneracy and the horizon decay of forecasts started from analyses should improve. This should hold without changing the filter, the ensemble size or the emulator architecture.

The training objective is

$$ \mathcal{L}(\theta) = \mathbb{E}_{x^a \sim \pi_{\theta'}}\big[\, \| \Phi_{\Delta}(x^a) - f_\theta^{(k)}(x^a) \|^2 \,\big], $$

The symbols are as follows. $\pi_{\theta'}$ is the distribution of analysis particles produced by running the filter with a frozen earlier emulator $f_{\theta'}$. $\Phi_{\Delta}$ is the true solver advanced over one assimilation window. $f_\theta^{(k)}$ is the emulator rolled out for the $k$ steps covering that window. The diagnostic we track is the local effective sample size, $\mathrm{ESS}_j = 1/\sum_i (w^i_j)^2$, where $w^i_j$ is the normalized weight of particle $i$ in local region $j$.

Relation to Prior Work

Several papers motivate the gap by showing that open-loop training fixes cannot stop chaotic divergence. HERO trains neural operators against a lagged checkpoint's failure modes and cuts 100-step error on Burgers from 0.541 to 0.153. It never sees observations. The WEATHER-5K benchmark shows that every data-driven forecaster deteriorates after about 72 hours. Both papers treat the rollout distribution, not the filter's analysis distribution, as the thing to train on. We change which distribution the emulator is trained on.

Another group of papers puts filtering and learned dynamics together but trains them differently. Ensemble latent assimilation runs an ensemble over a 16-dimensional autoencoder latent with an LSTM forecaster. The forecaster is trained offline and never on analysis states. Particle Filter Recurrent Neural Networks does train end to end through a particle filter. Its particles live in small hidden states, and its goal is sequence prediction rather than emulator fidelity on a physical field. Our training is not end-to-end backpropagation through weights. Instead, the solver labels the states the filter visits, which scales to spatial fields of size 64×64 and 256×256.

Generative assimilation methods are the strongest competitors and also constrain the design. DAISI maps forecasts backward into a pretrained flow model's noise space and resamples forward under observation guidance. It beats the Local Ensemble Transform Kalman Filter (LETKF) on Surface Quasi-Geostrophic (SQG) turbulence with 5% of the field observed, but it carries an uncorrected approximation bias. Control-Augmented Diffusion steers a frozen diffusion forecaster with an amortized controller. It relies on a preview window of future observations, so it is not causal filtering. Thermalizer projects drifting states back onto the invariant measure, the system's long-run distribution of states, and extends stable rollouts by up to 100×. It pulls toward a plausible state, not toward the observed one.

All three repair the state after the emulator has drifted. We instead remove the reason the emulator drifts on analysis inputs. The two approaches can be combined, and one experiment below tests the combination.

For the filters themselves we reuse existing methods. We use spatial localization in the style of Reich's ensemble transform work and LETKF. We avoid Stein-type particle flows (Maken et al.) as the primary filter. Understanding the Variance Collapse of SVGD shows their variance shrinks as $O(n/d)$ when particles are few relative to dimension, which would confound our ensemble-spread measurements.

Research Program

The work is organized around twin experiments, where synthetic observations are generated from a known true trajectory. In this setting the solver $\Phi$ is available at training time. This is the same condition under which emulators are built in the first place, so it adds no new assumption. We first train a baseline U-Net or Fourier neural operator emulator on clean solver trajectories. We then alternate two phases.

In the first phase, a localized filter is run with the current emulator on training trajectories, and analysis particles are stored. Two filters are used, a localized particle filter and LETKF, which yields two analysis distributions. In the second phase, each stored particle is advanced with the solver to produce labels, and the emulator is fine-tuned on a mix of clean and analysis-state pairs. A replay buffer keeps the emulator good on clean states.

Three to five rounds should suffice. We check this by tracking the shift between successive analysis distributions, measured as a distance between particle sets.

Alongside the method, we analyze the mechanism. We decompose the forecast–observation gap into a chaotic part and an emulator-error part. The chaotic part comes from running the solver on analysis states. The emulator-error part is the difference between the emulator and the solver from those same states. We then relate the emulator-error part to local effective sample size across observation densities and window lengths. If the core claim is right, the emulator-error share should dominate in sparse-observation, long-window settings, and it should fall after training on analysis states.

We will also test a weaker variant for settings with no solver available. In this variant the label is the true next state $x^*_{t+1}$ rather than $\Phi(x^a)$. We expect it to blur predictions, because it regresses an uncertain input onto a single answer. Measuring how much it blurs tells practitioners what solver access buys.

Baselines

The naive reference point is the clean-trained emulator dropped into the same filter, with ensemble size, localization radius and inflation tuned by grid search on validation trajectories. Every other method gets the same search budget. Any gain over this reference is the effect of the training distribution alone, since nothing else changes.

The strongest classical competitor is LETKF with tuned multiplicative inflation running on the clean emulator. Operationally, inflation is the standard response to model error. It may absorb much of the drift we target. We beat it only if training on analysis states lowers analysis RMSE and continuous ranked probability score (CRPS) at matched ensemble size after both are tuned. CRPS scores the whole ensemble forecast against the truth, not just its mean.

The strongest learned competitor is DAISI on SQG at 64×64 and 256×256 with 5% observation. Our method will be run with the same ensemble size and a matched wall-clock budget. The file gives no number to quote: no reported result found on Lacuna for DAISI's SQG RMSE or its SEVIR CRPS, only the qualitative statement that it beats LETKF in the sparse regime. We will rerun DAISI ourselves. The bar is matching its CRPS at a fraction of its per-cycle cost. That cost comes from its backward–forward stochastic integration, against our single forward pass per particle.

For open-loop context, a HERO-trained emulator is inserted into the same filter. Its reported 0.541 to 0.153 nRMSE@100 on Burgers (HERO) shows it is a strong rollout regularizer.

The key ablation replaces analysis states with true states plus Gaussian perturbations, labelled by the solver. The perturbations match the analysis-minus-truth error in variance and energy spectrum but have no filter-specific structure, such as localization seams or observation-footprint patterns. The second ablation feeds analysis states from LETKF to a surrogate deployed in the particle filter, and vice versa. It tests whether the benefit depends on the specific filter.

Why not just…?

Why not just train with input noise or pushforward rollouts? This is the most serious threat. If matched-spectrum Gaussian noise gives the same gain, the filter adds nothing beyond a noise schedule, and we would report that honestly. Our hypothesis says it will fall short in two places. The first is localized filters, because analysis increments carry seams and observation-footprint structure that isotropic noise lacks. The second is sparse masking, where increments are concentrated near sensors. The matched-noise ablation on all observation operators settles the question directly.

Why not just tune inflation in LETKF? Inflation widens the ensemble uniformly to cover model error. It cannot make the emulator's error smaller or less biased, so it should trade sharpness for coverage. That trade would show up as a worse spread-skill ratio, lower CRPS skill and damaged high-wavenumber energy. DAISI and the Gaussian-mixture transform filter also document LETKF and the ensemble Kalman filter failing under multimodal or saturating observation operators. We will test LETKF in both forms, with the clean emulator and with ours, since the two changes are complementary.

Why not just thermalize analysis states before forecasting? A Thermalizer-style projection would put analysis states back on the invariant manifold, but it does so by moving them away from the observations the filter just used. It also adds about 60% per-step cost for its drift classifier, and its thresholds need hand tuning. We run thermalized analyses as a baseline and expect lower spectral error but higher analysis RMSE. If thermalization matches us on RMSE, the idea reduces to manifold projection, and we will say so.

Why not just use DAISI or a diffusion assimilator? These methods are strong in sparse, multimodal regimes, but each cycle is expensive and their bias goes uncorrected. Our method is cheaper and orthogonal: an emulator trained on analysis states could serve as DAISI's forecast step. One run on SQG at 256×256 tests that combination.

Proposed Experiments and Decisive Checks

The benchmarks span a range of dimensions. Lorenz '96 at 40 to 400 dimensions, as used by Albarakati et al., is a cheap sanity check where degeneracy can be measured exactly. 2D Kolmogorov flow uses the four observation operators of Control-Augmented Diffusion, downsampled ×2 and ×4 and masked at stride 2 and 4, under strictly causal filtering. SQG at 64×64 and 256×256 is shared with DAISI. Two-layer quasi-geostrophic flow is a harder test for transfer.

Metrics are analysis RMSE, CRPS, spread-skill ratio, rank histograms, kinetic-energy-spectrum error, and the median and 10th-percentile local effective sample size. We also measure the longest observation interval at which analysis RMSE stays below observation noise. For horizon decay, we measure RMSE and spectral error of free forecasts launched from analyses, as a function of lead time.

Three results would decide the question. First, at fixed ensemble size, emulators trained on analysis states should raise the 10th-percentile local effective sample size and lengthen the sustainable observation interval on sparse SQG and masked Kolmogorov flow. They should also beat the matched-noise ablation by a margin larger than seed variance. If they do not, the central hypothesis fails. Second, the emulator-error share of the forecast–observation gap should fall after training. If effective sample size improves without this share falling, the mechanism we propose is wrong even if the method helps. Third, free forecasts from analyses should keep their energy spectra longer than those of the clean and HERO-trained emulators. This would show that the gain carries into the horizon-decay regime and is not confined to the filter.