Scientists increasingly plan experiments with pretrained policies. A policy is a neural network that, given the experiments run so far, proposes the next one. These policies are trained offline on millions of simulated experiments so that at deployment they choose in milliseconds instead of hours. Most such simulators assume clean, bell-curve (Gaussian) measurement noise. Real instruments often do not behave that way. Photon and molecule counts follow Poisson statistics, MRI magnitudes are skewed, and hyperparameter evaluations have occasional wild runs. Nobody knows how much this mismatch costs once the policy is judged by the decision it ultimately supports: which dose to give, which treatment to assign, which configuration to ship.
This note proposes to measure that cost in units practitioners care about, which we call the query tax. The query tax is the number of extra real experiments a policy trained on the wrong noise model needs before it matches a policy trained on the right one. The note then tests whether a cheap diagnostic, computed from the pretrained policy itself, can predict that tax before deployment. The central, falsifiable bet is directional. Policies that optimize for decisions concentrate their queries near the boundary where the best action flips, and a single heavy-tailed outlier near that boundary can flip the decision. We therefore expect decision-aware policies to be more fragile to noise misspecification than information-seeking ones. If so, "simulate with Gaussian noise and pretrain" is a worse default for goal-oriented design than for classical design.
Core Claim
Simulator fidelity for goal-oriented experimental design should be judged at the level of decisions, not likelihoods or sample statistics. The needed fidelity is highly uneven. Noise misspecification matters only where it changes which experiment is chosen or which action is taken. That unevenness can be estimated offline from the amortized policy's own sensitivity to noise perturbations.
Concretely, let $\pi_q$ be a policy pretrained on simulated rollouts with noise model $q$. Let $U_T(\pi_q; p)$ be the expected downstream utility after $T$ queries when real noise follows $p$. The query tax is
$$ \tau(q \to p) = \min\{\, k : U_{T+k}(\pi_q; p) \ge U_T(\pi_p; p) \,\}, $$
the number of additional real queries the mismatched policy needs to reach the utility the matched policy attains in $T$. It plays the same role for experimental design that the synthetic-to-real equivalency ratio plays for training data in On the Equivalency, Substitutability, and Flexibility of Synthetic Data. That paper found roughly 30 synthetic video frames worth one real frame for tracking. Our quantity is instead denominated in real experiments and downstream utility.
Where This Sits
The design side comes from amortized decision-aware experimental design. Amortized Decision-Aware Bayesian Experimental Design and its companion Amortized Bayesian Experimental Design for Decision-Making train a Transformer with two heads, the Transformer Neural Decision Process (TNDP). One head predicts experimental outcomes, and the other proposes the next design. The policy is trained on simulated rollouts to maximize Decision Utility Gain, which is how much an experiment raises the best achievable expected utility. They report 0.015 seconds per acquisition against 1558 seconds for targeted-EIG on a synthetic medical-diagnosis task. Their prediction head is Gaussian, and the rollouts assume well-behaved noise. We reuse their architecture and tasks and study exactly what they leave unexamined. ExPT: Synthetic Pretraining for Few-Shot Experimental Design similarly pretrains on Gaussian-process synthetic functions and also never varies the observation noise. Two further lines motivate why decision-awareness should change the noise question. Decision-Focused Sequential Experimental Design: A Directional Uncertainty-Guided Approach shows that only error directions relative to the decision geometry matter. Mean-objective-cost-of-uncertainty work (Experimental Design via Generalized Mean Objective Cost of Uncertainty) scores data by the decision gap it closes. Both assume the noise likelihood is known.
The robustness side constrains the design. Improving Robustness to Model Misspecification in Bayesian Experimental Design mixes the nominal model with a Gaussian-process fallback and shows that standard design can cluster measurements around false boundaries. Maximin Robust Bayesian Experimental Design derives a tempered posterior and Sibson's α-mutual information from a game against a KL-bounded adversary and warns that this can be over-conservative. Both are information-seeking, non-amortized, and tested on stylized models. Neither asks what misspecification does to decisions made by a pretrained policy.
The evaluation side comes from synthetic-data validation. Is this Model for Real? Simulating Data to Reveal the Proximity of a Model to Reality judges a simulator by whether real and simulated data land in the same regions of likelihood space. Design-Bench shows that optimizers exploit surrogate errors even when the surrogate looks reasonable. Together they suggest that likelihood-level authenticity and decision-level adequacy can come apart. Our diagnostic tests this directly.
Research Program
The first component is a measurement study. We pretrain TNDP policies under a Gaussian simulator and deploy them under controlled true noise. The true noise families are Student-t with varying degrees of freedom, Poisson counts at varying intensity, and multiplicative noise. We measure the query tax on each task. As a matched reference, we pretrain an information-seeking amortized policy on identical rollouts by swapping the Decision Utility Gain reward for predictive information gain. This isolates whether decision-awareness itself changes fragility. The hypothesis is that decision-aware policies pay a larger tax under heavy tails and a smaller one under noise that is symmetric and light-tailed away from the decision boundary. If the tax is uniformly small, the conclusion is also useful: Gaussian simulation would be safe for this class of methods.
The second component is the diagnostic. For a pretrained policy and a candidate noise family, we compute a decision sensitivity score. We replay simulated histories under the alternative noise and record two quantities: how often the chosen design changes, and how much the final action's utility drops, weighted by the policy's own predicted utility. This needs only simulation, no real data. We compare it against two decision-blind fidelity measures as predictors of the true query tax across task and noise pairs. The first is the likelihood-topography distance of Is this Model for Real?. The second is the prediction head's generic calibration error. We also test a deployment-time variant, a calibration check on the prediction head computed only on outcomes near the current decision boundary, as an early warning.
The third component asks whether a straightforward fix removes the tax. The fix pretrains over a prior that includes the noise family as a latent variable and replaces the Gaussian prediction head with a flexible one. We use a discretized-histogram output, which can represent counts, skew, and heavy tails. Because Decision Utility Gain is defined over the full prior, such a policy should spend queries identifying the noise family only when that identification moves the decision. This is ordinary goal-oriented design applied to the noise model. The open question is out-of-family noise, which no pretraining prior contains.
The tasks follow the file. The first two are TNDP's 1D decision-aware regression and the synthetic medical-diagnosis active-learning task with four treatments. In the diagnosis task we re-express lab measurements as counts, so that Poisson noise is natural. We also use the Michaelis–Menten/Hill kinetics setup from the misspecification paper. Finally, we use HPO-B on the ranger, svm, rpart and xgboost meta-datasets that TNDP used. HPO-B provides our one source of genuinely non-synthetic, out-of-family noise. We fit the benchmark's XGBoost surrogate and resample its empirical residuals against the raw OpenML evaluations as the "real" noise. Its tails and skew are whatever the data say, not a family we chose.
Baselines
The naive reference point is the unmodified Gaussian-pretrained TNDP deployed as-is. Every query tax is measured relative to it, and a random-design policy bounds performance from below.
The strongest existing amortized method is TNDP itself, retrained with oracle knowledge of the true noise. This is the matched policy in the tax definition, and it upper-bounds what any fix can achieve. Its reported figures are 0.015 seconds per acquisition and 0.31 seconds for 10 rounds on the diagnosis task, versus 1558 seconds per step for targeted-EIG and over 5,700 seconds for 10 rounds of GP-DEIG. On decision accuracy and HPO-B utility, the paper states qualitative wins over UCB, EI, PI and PFNs4BO, but no reported numerical result found on Lacuna for TNDP's decision accuracy or HPO-B utility gains. We will therefore rerun TNDP from scratch rather than compare against quoted numbers.
The strongest robustness competitor is the non-amortized α-tempered design of Maximin Robust Bayesian Experimental Design, adapted to a decision objective by tempering the likelihood inside the expected-utility computation. We also run the mixture-with-Gaussian-process-fallback design of Improving Robustness to Model Misspecification. Both receive the same query budget. Because they are slow, we give them a wall-clock budget matched to TNDP's pretraining cost amortized over the evaluation episodes, and we tune α and the mixture weight on held-out simulated tasks.
Ablations separate the two ingredients of the fix. One variant keeps the Gaussian head and adds only noise-family randomization. The other adds only the flexible head. All policies share rollout count, architecture, REINFORCE budget, and tuning trials.
For the fix, beating the baseline means reducing the median query tax to within one query of the matched oracle on in-family noise, with no utility loss under true Gaussian noise. For the diagnostic, it means a clearly higher rank correlation with the measured tax than both decision-blind fidelity measures across all task and noise pairs.
Why not just…?
Why not just randomize the noise during pretraining and stop there? This is the most serious threat. Our own reasoning says it should largely work for in-family noise, because a prior over noise families makes resolving noise part of the decision problem. We do not argue around it; it is the method in component three, and the ablations measure how much of it is needed. If it closes the tax everywhere, including on HPO-B's empirical residuals, the paper's contribution reduces to the measurement result and the diagnostic. The diagnostic would still tell practitioners which families are worth including. The experiment that settles this is the out-of-family HPO-B condition.
Why not just swap in a Student-t or Poisson likelihood with classical information-gain design? Goal-Oriented Bayesian Optimal Experimental Design for Nonlinear Models using MCMC shows that parameter-level information gain picks different, worse designs when the quantity of interest relates non-bijectively to parameters. Nested estimators with non-Gaussian likelihoods are also slow, as the 1558-second targeted-EIG step shows. Our matched information-seeking amortized policy tests this cleanly. It receives the correct likelihood, and we measure whether it still loses decision utility.
Why not just clip outliers and keep the Gaussian model? Clipping helps when outliers are pure corruption, and normalization alone made gradient ascent competitive on Design-Bench. However, Is this Model for Real? found that apparently extreme fits reflected genuine behavior. In count data, the low counts that clipping distorts are exactly the ones near a dose threshold. We include a clip-and-Gaussian variant of TNDP as an additional baseline. If it matches the fix, that is a practically important finding, and we will report it as such.
Why not just fine-tune on a few real experiments? In synthetic-data work, fine-tuning is what closes the gap: 20% real data nearly matched full real training in On the Equivalency, Substitutability, and Flexibility of Synthetic Data. In experimental design, however, the real experiments are the scarce resource being budgeted. The query tax is precisely the cost of fine-tuning in real queries, so this alternative is what we measure rather than a competitor.
Decisive Checks
Three results decide the proposal. The first is the sign of the fragility comparison: does the decision-aware policy pay a larger query tax than the information-seeking one under Student-t and Poisson noise at matched budgets? The second is whether the simulation-only decision sensitivity score predicts measured tax better than likelihood-topography distance and generic calibration. It must do so on held-out tasks, not only on the tasks used to design it. The third is whether noise-randomized pretraining with a flexible head leaves a residual tax on HPO-B's empirical residual noise. A residual would show that noise families chosen for mathematical convenience do not stand in for real instruments, even when the policy is trained to hedge over them. A large tax that the score predicts, together with a residual on real residuals, would establish that goal-oriented design needs decision-level simulator validation. A null result on the first check would establish the opposite: Gaussian simulation is safe for this class of methods.
References
- On the Equivalency, Substitutability, and Flexibility of Synthetic Data
- Amortized Decision-Aware Bayesian Experimental Design
- Amortized Bayesian Experimental Design for Decision-Making
- ExPT: Synthetic Pretraining for Few-Shot Experimental Design
- Decision-Focused Sequential Experimental Design: A Directional Uncertainty-Guided Approach
- Experimental Design via Generalized Mean Objective Cost of Uncertainty
- Improving Robustness to Model Misspecification in Bayesian Experimental Design
- Maximin Robust Bayesian Experimental Design
- Is this Model for Real? Simulating Data to Reveal the Proximity of a Model to Reality
- Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization
- HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML
- Goal-Oriented Bayesian Optimal Experimental Design for Nonlinear Models using Markov Chain Monte Carlo