Regression Based Summary Statistics for Approximate Bayesian Computation

This direction is merged into another direction: likelihood-free inference (intractable likelihood functions). Its last version stays readable and citable here.

Version 1 of 111 papers16 conceptsfrozen 2026-10-04

In Bayesian inference, we often face a "likelihood-free" problem: we can simulate data from a complex model, but we cannot write down the mathematical formula for the probability of that data. Approximate Bayesian Computation (ABC) solves this by comparing simulated data to observed data. However, as Dennis Prangle notes in Summary Statistics in Approximate Bayesian Computation, ABC suffers from the "curse of dimensionality." If you compare raw, high-dimensional datasets directly, the probability of a simulation landing near the observation becomes vanishingly small.

To make ABC work, researchers must condense raw data into a few highly informative "summary statistics." Historically, these were hand-picked by experts, but a more powerful approach has emerged: using regression to learn the optimal summaries automatically.

The Intuition: The Posterior Mean as the Best Summary

The core breakthrough in this field, pioneered by Paul Fearnhead in Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC, is the realization that the posterior mean of a parameter is the theoretically optimal summary statistic.

If your goal is to estimate a parameter $\theta$, the most informative single value you can extract from your data is your best guess of $\theta$ itself. By using the estimated posterior mean as a summary statistic, you project high-dimensional data into a low-dimensional space—specifically, a space with the same dimensionality as your parameters—while preserving the maximum amount of relevant information.

The Semi-Automatic Mechanism

Since we don't know the true posterior mean (that is what we are trying to find), we estimate it using a "semi-automatic" two-step process:

  1. Pilot Simulations: Generate a large set of simulated parameters and their corresponding data.
  2. Regression Training: Train a regression model where the simulated data features are the predictors and the parameters are the targets. As Matthew A. Nunes explains in the documentation for abctools: An R Package for Tuning Approximate Bayesian Computation Analyses, this model learns a mapping from the complex data space to the parameter space.

Once trained, this regression model acts as a "distiller." When you perform the actual ABC inference, you pass both your observed data and your new simulations through this model. The resulting predictions serve as the summary statistics. This ensures the statistics are explicitly optimized to distinguish between different parameter values.

From Linear Models to Machine Learning

The "semi-automatic" label originally referred to using linear regression on transformed data features. Paul Fearnhead showed that even simple linear models can capture complex posterior relationships if the data is pre-processed correctly.

However, the framework has expanded to more sophisticated models. Yannik Schälte and the team behind pyABC: Efficient and robust easy-to-use approximate Bayesian computation have integrated machine learning models to automatically extract features from raw data. In more theoretical directions, Dino Sejdinovic introduced DR-ABC: Approximate Bayesian Computation with Kernel-Based Distribution Regression, which uses kernel methods to represent data distributions directly, bypassing the need for manual feature engineering entirely.

Applications in Complex Systems

This regression-based approach is particularly effective for models where the "signal" is buried in noisy, high-dimensional output:

By transforming the problem of "choosing statistics" into a problem of "predicting parameters," regression-based ABC provides a principled, data-driven way to perform inference on the world's most complex generative models.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_regression_based_summary_statistics_for__12cd19e0,
  title = {Regression Based Summary Statistics for Approximate Bayesian Computation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/regression-based-summary-statistics-for-approximate-bayesian-computation/rdr_12cd19e04cff4083a689fb67c5331432},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_regression_based_summary_statistics_for__12cd19e0_v1,
  title = {Regression Based Summary Statistics for Approximate Bayesian Computation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/regression-based-summary-statistics-for-approximate-bayesian-computation/rdr_12cd19e04cff4083a689fb67c5331432/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}