Summary Statistics for Approximate Bayesian Computation

This direction is merged into another direction: likelihood-free inference (intractable likelihood functions). Its last version stays readable and citable here.

Version 1 of 126 papers34 conceptsfrozen 2026-10-04

In many scientific fields, we have perfect simulators but impossible math. We can simulate the evolution of a galaxy or the spread of a virus, but we cannot write down the "likelihood function"—the mathematical formula for the probability of observing our specific data given a set of parameters.

Approximate Bayesian Computation (ABC) is the standard tool for these "likelihood-free" scenarios. As Paul Fearnhead explains, ABC works by a simple "simulate-and-compare" logic: pick a parameter, run the simulator, and if the simulated data looks like the real data, keep that parameter (Constructing ABC summary statistics: semi-automatic ABC).

However, this simplicity hides a fundamental geometric trap: the curse of dimensionality.

The Curse of Dimensionality

If you try to compare raw, high-dimensional datasets directly, ABC almost always fails. In high-dimensional space, everything is far apart. The probability that a simulation will exactly match a complex dataset is effectively zero. As Dennis Prangle notes, acceptance rates in ABC decrease exponentially as the number of dimensions increases (Summary Statistics in Approximate Bayesian Computation).

To make the comparison feasible, we must condense the data into a few summary statistics—numbers like the mean, variance, or specific spatial correlations. But this creates a brutal trade-off:

  1. Too few statistics: You lose critical information. Different parameters might produce the same summary values, leading to biased or overly broad estimates.
  2. Too many statistics: You trigger the curse of dimensionality. The "noise" from the extra dimensions makes it impossible for simulations to ever get "close" to the observed data.

Robert J. Erhardt points out a counter-intuitive result of this tension: high-dimensional statistics that are theoretically "sufficient" (containing all possible information) often perform worse than low-dimensional, insufficient ones because the distance-based matching process breaks down in high dimensions (Modelling extremes using approximate Bayesian Computation).

Semi-Automatic Selection

The "black art" of ABC has traditionally been the manual selection of these statistics. To move beyond hand-picked summaries, Paul Fearnhead and Dennis Prangle pioneered "semi-automatic ABC." Their insight was to use a pilot run of simulations to learn a regression model that maps data to parameters. The predicted parameters themselves then become the summary statistics, effectively projecting high-dimensional data onto the exact number of dimensions as the parameter space (Constructing ABC summary statistics: semi-automatic ABC).

This direction has evolved into using deep learning to "learn" the best representations. For example, Michael U. Gutmann and Traiko Dinev use Convolutional Neural Networks (CNNs) to automatically extract features from raw time-series data, bypassing manual feature engineering (Dynamic Likelihood-free Inference via Ratio Estimation (DIRE)). Similarly, Carlo Albert and colleagues have used modified autoencoders to distill parameter-relevant information while ignoring random simulation noise (Learning Summary Statistics for Bayesian Inference with Autoencoders).

Bypassing Summaries with Statistical Distances

A different approach to the problem is to stop using summary statistics entirely. Instead of condensing data into a few numbers, researchers like Christopher Drovandi and David T. Frazier compare the full empirical distributions of the data using robust statistical distances like the Wasserstein distance or Cramer-von Mises distance (A comparison of likelihood-free methods with and without summary statistics).

These "full data" methods aim to preserve all information without the information loss inherent in summarization. However, as Drovandi and Frazier demonstrate, these methods are not a silver bullet; their performance is highly sensitive to how the data is scaled and transformed, suggesting that the fundamental challenge of data representation remains the central hurdle in likelihood-free inference (A Comparison of Likelihood-Free Methods With and Without Summary Statistics).

Localizing the Solution

Recent refinements focus on making the dimension reduction "local." Jukka Sirén and Samuel Kaski argue that we don't need a global mapping of data to parameters. Instead, we only need to understand the relationship between statistics and parameters in the immediate neighborhood of our observed data (Local dimension reduction of summary statistics for likelihood-free inference). By focusing the computational effort where it matters most, these methods further mitigate the curse of dimensionality, allowing for accurate inference in increasingly complex models.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_summary_statistics_for_approximate_bayes_2e53c4e5,
  title = {Summary Statistics for Approximate Bayesian Computation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/summary-statistics-for-approximate-bayesian-computation/rdr_2e53c4e52e564b109ff23dbcd5dbf6e6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_summary_statistics_for_approximate_bayes_2e53c4e5_v1,
  title = {Summary Statistics for Approximate Bayesian Computation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/summary-statistics-for-approximate-bayesian-computation/rdr_2e53c4e52e564b109ff23dbcd5dbf6e6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}