Bayesian inference is a cornerstone of scientific modeling, offering a mathematically rigorous way to update our beliefs about model parameters $\theta$ in light of observed data $x_{\text{obs}}$. At its heart lies Bayes' theorem:
$$p(\theta | x_{\text{obs}}) = \frac{p(x_{\text{obs}} | \theta) p(\theta)}{p(x_{\text{obs}})}$$
To compute the posterior distribution $p(\theta | x_{\text{obs}})$, we must evaluate the likelihood function $p(x_{\text{obs}} | \theta)$—the probability density of observing our exact data given a specific parameter set.
However, modern scientific models are frequently implemented as complex, stochastic computer simulations rather than closed-form equations. In fields ranging from epidemiology (Warne et al. 2020) and tumor dynamics (Wang et al. 2024) to population genetics (Estoup et al. 2004), these simulators involve countless latent variables, random branching paths, and multi-scale interactions. While it is straightforward to run the simulator forward to generate synthetic data, calculating the analytical probability of generating a specific dataset is computationally impossible.
When the likelihood function is intractable, traditional Bayesian tools break down. Approximate Bayesian Computation (ABC) solves this dilemma by replacing formal likelihood evaluations with forward simulations (Lintusaari 2016).
Bypassing the Likelihood via Forward Simulation
The core intuition behind ABC is surprisingly simple: if a candidate parameter set $\theta^*$ is highly likely to have generated the observed data, then running the simulator with $\theta^*$ should frequently produce synthetic data that looks exactly like the observed data.
Consider the foundational ABC Rejection Algorithm outlined by Filippi et al. 2013:
- Sample a candidate parameter vector $\theta^*$ from the prior distribution $p(\theta)$.
- Plug $\theta^*$ into the simulator to generate a synthetic dataset $x_{\text{sim}}$.
- If $x_{\text{sim}} = x_{\text{obs}}$, accept $\theta^*$; otherwise, reject it.
Why does this work? By sampling $\theta^*$ from the prior and then sampling $x_{\text{sim}}$ from the simulator, we are implicitly drawing a joint sample $(\theta^*, x_{\text{sim}})$ from the joint distribution $p(\theta, x) = p(x | \theta)p(\theta)$. By filtering these samples and keeping only those where $x_{\text{sim}} = x_{\text{obs}}$, the accepted parameters are guaranteed to be exact draws from the true posterior distribution $p(\theta | x_{\text{obs}})$. We have performed Bayesian inference without ever writing down or calculating a likelihood function.
The Practical Compromise: Summary Statistics and Tolerance
While the rejection algorithm is theoretically perfect, it encounters an immediate practical barrier: the curse of dimensionality. For continuous or high-dimensional datasets, the probability that a stochastic simulator generates a dataset exactly identical to the real-world data is effectively zero. The acceptance rate drops to zero, and the algorithm stalls.
To make ABC viable, we must introduce two distinct approximations:
- Summary Statistics ($S$): Instead of comparing raw, high-dimensional datasets, we compress both the observed and simulated data into low-dimensional summary statistics, $S(x_{\text{obs}})$ and $S(x_{\text{sim}})$, which capture the essential features of the system.
- Tolerance Threshold ($\epsilon$): Instead of demanding a perfect match, we accept a parameter set if the distance $\rho$ between the simulated and observed summary statistics falls within a small tolerance threshold:
$$\rho\left(S(x_{\text{sim}}), S(x_{\text{obs}})\right) \le \epsilon$$
This relaxation transforms the inference problem. As Gadat et al. 2015 demonstrate, this formulation can be mathematically framed as a $k$-nearest neighbor regression problem. The choice of $\epsilon$ dictates a fundamental trade-off: as $\epsilon \to 0$, the approximation approaches the true posterior but requires an infinite number of simulations; as $\epsilon$ increases, computational efficiency rises, but the resulting distribution becomes smoother and wider than the true posterior, introducing bias.
[Prior Distribution p(θ)]
│
▼
Sample Parameter θ*
│
▼
┌─────────────────────────┐
│ Black-Box Simulator │ ──► Generates Synthetic Data (x_sim)
└─────────────────────────┘
│
▼
Compute Summary Statistic S(x_sim)
│
▼
Is Distance ρ(S(x_sim), S(x_obs)) ≤ ε?
╱ ╲
YES NO
╱ ╲
▼ ▼
[Accept θ*] [Reject θ*]
Scaling Up with Sequential Sampling
The simple rejection sampler is highly inefficient because it continuously draws samples from the prior distribution. If the prior is wide and the posterior is narrow, the vast majority of simulations will be wasted on parameter regions that yield poor matches.
To optimize this process, researchers embed ABC within iterative sampling frameworks. One of the most powerful variants is Sequential Monte Carlo ABC (SMC-ABC). Instead of targeting a single, tiny tolerance threshold $\epsilon$ immediately, SMC-ABC maintains a population of parameter "particles" and guides them through a sequence of decreasing thresholds $\epsilon_1 > \epsilon_2 > \dots > \epsilon_T$.
In each iteration, particles are sampled from the previous, wider distribution, perturbed slightly, and tested against the new, stricter threshold. This allows the algorithm to progressively focus its computational budget on high-probability regions of the parameter space.
This sequential approach has unlocked parameter estimation for highly complex dynamical systems where traditional analysis is intractable. For example:
- Cellular Biology: Carr et al. 2021 utilized SMC-ABC to calibrate stochastic cell invasion models, tracking how individual melanoma cells move and divide.
- Neuroscience: Samson et al. 2025 applied it to the stochastic FitzHugh-Nagumo model to successfully estimate neural firing parameters from real rat action potential recordings.
- Sociolinguistics: Zhou 2020 leveraged SMC-ABC to estimate latent social and economic parameters governing language competition and extinction trends over decades.
- Epidemiology: Stumpf 2011 demonstrated how sequential ABC engines can successfully transition qualitative biological observations into rigorous, quantitative mechanistic models.
Beyond Distance Metrics
While standard ABC relies heavily on choosing an appropriate distance metric $\rho$ and tolerance $\epsilon$, alternative formulations seek to bypass these choices entirely.
One prominent alternative is introduced by Chaudhuri 2024, who proposes integrating empirical likelihood into the ABC framework (ABCel). Instead of filtering simulations via an arbitrary binary threshold ($\le \epsilon$), this approach uses non-parametric empirical likelihood to weight simulations based on how well they match the data's statistical properties. This provides a statistically rigorous, data-driven solution that is particularly effective for complex, dependent data structures where standard distance metrics fail to capture underlying correlations.
Key Foundational Work
GENETIC ANALYSIS OF COMPLEX DEMOGRAPHIC SCENARIOS: SPATIALLY EXPANDING POPULATIONS OF THE CANE TOAD, BUFO MARINUS (2004) — contributed 2 concept elements to this direction.
COMPUTER PROGRAMS: <scp>on</scp>e<scp>samp</scp>: a program to estimate effective population size using approximate Bayesian computation (2007) — contributed 2 concept elements to this direction.