Amortised and provably-robust simulation-based inference

Ayush Bharti, Charita Dellaporta, Yuga Hikida, François-Xavier Briol

Abstract

Complex simulator-based models are now routinely used to perform inference across the sciences and engineering, but existing inference methods are often unable to account for outliers and other extreme values in data which occur due to faulty measurement instruments or human error. In this paper, we introduce a novel approach to simulation-based inference grounded in generalised Bayesian inference and a neural approximation of a weighted score-matching loss. This leads to a method that is both amortised and provably robust to outliers, a combination not achieved by existing approaches. Furthermore, through a carefully chosen conditional density model, we demonstrate that inference can be further simplified and performed without the need for Markov chain Monte Carlo sampling, thereby offering significant computational advantages, with complexity that is only a small fraction of that of current state-of-the-art approaches.

Introduction

Simulation-Based Inference (SBI) has become a cornerstone for analyzing complex scientific phenomena where the underlying likelihood is intractable. From epidemiology to cosmology, researchers rely on simulators to bridge the gap between theoretical models and observed data. A dominant paradigm in SBI is Neural Likelihood Estimation (NLE), which trains neural density estimators to approximate the simulator's likelihood. While NLE is powerful and "amortised"—allowing for rapid inference on new datasets after a single training phase—it is notoriously fragile. If the observed data contains outliers or is misspecified relative to the simulator, NLE often produces overconfident and highly biased posterior estimates.

Methodology: Neural Score-Matching Bayes

To address this fragility, we propose Neural Score-Matching Bayes (NSM-Bayes). This approach leverages Generalised Bayesian Inference (GBI), which replaces the standard likelihood with a robust loss function. Specifically, we utilize a weighted score-matching loss to update beliefs.

The process involves two primary steps:

  1. Surrogate Training: A flexible neural density model (such as a Normalising Flow) is trained on clean, simulated data to learn the score function—the gradient of the log-likelihood.
  2. Weighted Inference: Upon receiving real-world data, the posterior is updated using a weighted score-matching objective. A weight function $w(x)$ acts as a filter, identifying and down-weighting data points that deviate significantly from the simulator's expected output (outliers).

The Conjugate Advantage

We further introduce NSM-Bayes-conj, a variant that utilizes a specific exponential family for the neural model. When paired with a Gaussian prior, this setup creates a conjugate relationship, allowing the posterior to be calculated in closed form. This innovation bypasses the need for computationally expensive Markov Chain Monte Carlo (MCMC) sampling, enabling near-instantaneous inference.

Robustness and Performance

Our framework provides formal mathematical guarantees of robustness. While we prove that standard NLE is not globally bias-robust, we demonstrate that NSM-Bayes, when used with specific weight functions like Inverse Multi-Quadratic weights, is guaranteed to remain stable even under infinitesimal data contamination.

Posterior distributions for NLE (blue) and NSM-Bayes (red) on data with outliers. The inset shows the data distribution with a significant outlier and the corresponding weight function $w(x)$ that down-weights it.

In empirical evaluations across benchmarks like the g-and-k distribution and the SIR epidemiology model, NSM-Bayes consistently outperformed standard methods. As shown in the figure above, while NLE's posterior estimates (blue) are pulled far from the true parameters by outliers, NSM-Bayes (red) remains centered on the ground truth. In the g-and-k task, NSM-Bayes achieved 100% empirical coverage, whereas NLE and other robust baselines failed completely. Furthermore, the conjugate version reduced inference time from hundreds of seconds to just 5 seconds.

Conclusion

NSM-Bayes represents a significant step forward in making SBI reliable for real-world applications where data is rarely perfect. By combining amortisation with provable robustness and MCMC-free inference, it offers a scalable solution for complex scientific modeling. Future work will focus on extending these principles to discrete data and addressing more fundamental forms of model misspecification beyond outliers.

Summary updated

Abstract

Complex simulator-based models are now routinely used to perform inference across the sciences and engineering, but existing inference methods are often unable to account for outliers and other extreme values in data which occur due to faulty measurement instruments or human error. In this paper, we introduce a novel approach to simulation-based inference grounded in generalised Bayesian inference and a neural approximation of a weighted score-matching loss. This leads to a method that is both amortised and provably robust to outliers, a combination not achieved by existing approaches. Furthermore, through a carefully chosen conditional density model, we demonstrate that inference can be further simplified and performed without the need for Markov chain Monte Carlo sampling, thereby offering significant computational advantages, with complexity that is only a small fraction of that of current state-of-the-art approaches.
BibTeX
@misc{bharti2026amortised,
  author = {Ayush Bharti and Charita Dellaporta and Yuga Hikida and François-Xavier Briol},
  title = {{Amortised and provably-robust simulation-based inference}},
  year = {2026},
  eprint = {2602.11325},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.11325},
}
Figure