Some discussions of D. Fearnhead and D. Prangle's Read Paper "Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation"

Christophe Andrieu, Simon Barthelmé, Nicolás Chopin, Julien Cornebise, Randal Douc, Mark Girolami, Ioannis Kosmidis, Ajay Jasra, Anthony Lee, Jean‐Michel Marin, Pierre Pudlo, Christian P. Robert, Mohammed Sedki, Sumeetpal S. SingharXiv 2012 · DOI · arXiv

Abstract

Approximate Bayesian Computation (ABC) has become a cornerstone for inference in models with intractable likelihoods. However, its performance is critically dependent on the choice of summary statistics. This report synthesizes a collection of expert commentaries on Fearnhead and Prangle’s (2011) landmark work regarding "semi-automatic" summary statistic selection, exploring diverse perspectives on latent variable representations, summary-free methods, and post-processing refinements.

The Challenge of Intractable Likelihoods

In many complex fields, such as population genetics or ecology, the likelihood function is often impossible or too computationally expensive to calculate. ABC addresses this by simulating data from the model and retaining parameters that produce simulated data close to the observed data. To manage high-dimensional datasets, researchers typically reduce data to a set of summary statistics. The fundamental challenge lies in selecting these statistics automatically to avoid introducing bias or losing vital information.

Methods and Perspectives

This collection of discussions proposes several distinct shifts in the ABC framework:

  • Latent Variable Representation: Andrieu et al. suggest that because simulations rely on underlying random variables (seeds), these should be treated as latent variables. This allows for more efficient sampling using Particle MCMC tools.
  • Summary-free ABC: Barthelmé et al. advocate for methods like Expectation Propagation (EP-ABC) and HMM-ABC, which bypass summary statistics entirely by leveraging specific model structures to approximate the posterior directly.
  • Post-processing Selection: Sedki and Pudlo propose using the Bayesian Information Criterion (BIC) during a regression step to prune irrelevant summary statistics after the initial simulation phase.
Graphical representations of ABC frameworks, comparing standard proxy-based approaches with auxiliary variable formulations.

Key Insights: ABC as a "Noisy" Model

A profound conceptual shift suggested in the discussion is viewing ABC not as a poor approximation of the true model, but as exact inference on a "noisy" version of the model where data is corrupted by a specific kernel. This perspective helps clarify the bias-variance tradeoff in statistic selection. Furthermore, while the "curse of dimensionality" suggests a minimal set of statistics, practical experience often shows that a larger, well-chosen set can provide a superior approximation of the true posterior.

Key Results

The efficacy of these refinements is demonstrated through various test cases. For instance, in a 20-dimensional Gaussian toy model, Sedki and Pudlo showed that using BIC to remove irrelevant statistics significantly improved the accuracy of the posterior density estimate and model selection compared to standard ABC approaches.

Impact of BIC on posterior density estimates (a) and model selection (b) compared to true values.

Limitations and Future Work

Despite these advances, several hurdles remain:

  • Pilot Run Dependency: The semi-automatic approach relies on a "pilot run" to estimate statistics. If this initial run is poor, the resulting statistics will also be suboptimal.
  • Model Specificity: Many "summary-free" methods are currently limited to specific model classes where data can be decomposed sequentially.
  • Quantifying Bias: It remains difficult to easily monitor or quantify the bias induced by a specific set of "optimal" summary statistics without extensive numerical comparisons.

Conclusion

The discussion highlights that while Fearnhead and Prangle provided a foundational framework for automating ABC, the field is rapidly evolving toward more integrated and theoretically grounded methods for handling the intractable likelihoods of modern statistical models.

Summary updated

Abstract

This report is a collection of comments on the Read Paper of Fearnhead and Prangle (2011), to appear in the Journal of the Royal Statistical Society Series B, along with a reply from the authors.
BibTeX
@misc{andrieu2012some,
  author = {Christophe Andrieu and Simon Barthelmé and Nicolás Chopin and Julien Cornebise and Randal Douc and Mark Girolami and Ioannis Kosmidis and Ajay Jasra and Anthony Lee and Jean‐Michel Marin and Pierre Pudlo and Christian P. Robert and Mohammed Sedki and Sumeetpal S. Singh},
  title = {{Some discussions of D. Fearnhead and D. Prangle's Read Paper "Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation"}},
  year = {2012},
  eprint = {1201.1314},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1201.1314},
}
Figure