Simulation Based Inference for Intractable Likelihoods

This direction is merged into another direction: likelihood-free inference (intractable likelihood functions). Its last version stays readable and citable here.

Version 1 of 1149 papers155 conceptsfrozen 2026-10-04

In fields ranging from cosmology to cognitive science, researchers often build models that are easy to run but impossible to "solve." These models, known as simulators, can generate realistic synthetic data given a set of parameters, but they lack a closed-form likelihood function—the mathematical formula that tells us the probability of observing specific data given those parameters.

Without a tractable likelihood, traditional Bayesian inference tools like Markov Chain Monte Carlo (MCMC) fail. Simulation-Based Inference (SBI), also known as likelihood-free inference, provides a way to reverse-engineer these "black-box" models, allowing scientists to estimate parameters and quantify uncertainty using only the simulator's outputs.

The Likelihood Wall

In a standard Bayesian workflow, we update our prior beliefs about parameters $\theta$ using the likelihood $p(x|\theta)$ to arrive at a posterior $p(\theta|x)$. However, as Hans R. Kuensch notes, complex models in genetics and other domains often hit a "likelihood wall." The internal stochasticity of the simulator—the thousands of random coin flips that happen during a weather simulation or a biological process—makes it impossible to write down an explicit density function.

As Jan-Matthis Lueckmann and Jakob H. Macke explain in Benchmarking Simulation-Based Inference, this intractability forces us to move from evaluating formulas to comparing samples.

From Rejection to Synthetic Likelihoods

The earliest solution to this problem was Approximate Bayesian Computation (ABC). The intuition is simple: if we run the simulator with parameters $\theta^*$ and the resulting synthetic data $x^*$ is "close enough" to our real observation $x_{obs}$, then $\theta^*$ is a plausible sample from the posterior.

To make this practical, researchers like Shigeki Nakagome and Samuel Kaski use "summary statistics" to reduce high-dimensional data into manageable chunks. However, ABC suffers from the "curse of dimensionality"—as the data becomes more complex, the chance of a simulator run landing "close" to the observation becomes vanishingly small.

To bridge this gap, Christopher Drovandi and colleagues developed Bayesian Synthetic Likelihood (BSL). Instead of simple rejection, BSL runs the simulator multiple times for a single $\theta$, assumes the resulting summary statistics follow a distribution (often a Gaussian), and estimates the parameters of that distribution to create a "synthetic" likelihood. This approach, detailed in BSL: An R Package for Efficient Parameter Estimation, allows for more robust inference in high-dimensional settings.

The Neural Revolution: Amortized Inference

Modern SBI has moved toward leveraging deep learning to learn the relationship between parameters and data directly. This is often called amortized inference: we spend a large amount of computational effort training a neural network on millions of simulations so that, once trained, we can perform inference on new observations almost instantly.

There are three primary neural strategies:

  1. Neural Posterior Estimation (NPE): Training a normalizing flow or mixture density network to directly output the posterior distribution $p(\theta|x)$.
  2. Neural Likelihood Estimation (NLE): Training a network to approximate the simulator's likelihood $p(x|\theta)$, which can then be used inside a standard MCMC sampler.
  3. Neural Ratio Estimation (NRE): Framing inference as a classification task. As Conor Durkan and George Papamakarios demonstrate in On Contrastive Learning for Likelihood-free Inference, a classifier trained to distinguish between "real" (paired) and "fake" (unpaired) parameter-data sets implicitly learns the likelihood ratio, which is sufficient for Bayesian inference.

Advanced Geometries and Optimal Transport

Recent work has pushed SBI beyond simple density estimation. For simulators that are differentiable, Amos Storkey and Matthew M. Graham proposed treating observations as hard geometric constraints. By using Constrained Hamiltonian Monte Carlo (CHMC), they sample directly from the manifold of inputs that could have produced the observed data, providing asymptotically exact inference.

Alternatively, researchers like Amirhossein Taghvaei and Ricardo Baptista have applied Optimal Transport (OT) to the problem. In this framing, inference is seen as finding a "map" that pushes a simple prior distribution toward the complex posterior. This avoids the weight degeneracy issues common in traditional particle filters and allows for efficient filtering in high-dimensional state spaces.

Tools for the Scientific Community

The maturation of this field is evidenced by the development of robust software ecosystems. The sbi toolkit, developed by Álvaro Tejero-Cantero and others, provides a unified PyTorch-based interface for these methods. These tools allow scientists to move from "black-box" simulation to rigorous statistical discovery, enabling the reverse-engineering of systems as diverse as biochemical networks and cosmological structures.

Papers in this version

Showing 60 of 149 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_simulation_based_inference_for_intractab_916a3309,
  title = {Simulation Based Inference for Intractable Likelihoods},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/simulation-based-inference-for-intractable-likelihoods/rdr_916a33091d694814923f67977e0bdddf},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_simulation_based_inference_for_intractab_916a3309_v1,
  title = {Simulation Based Inference for Intractable Likelihoods},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/simulation-based-inference-for-intractable-likelihoods/rdr_916a33091d694814923f67977e0bdddf/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}