Gaussian process modelling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria

Marko Järvenpää, Michael U. Gutmann, Aki Vehtari, Pekka Marttinen

TL;DR

This paper improves the efficiency of Approximate Bayesian Computation (ABC) by using Gaussian Processes (GPs) to surrogate expensive simulations. The authors demonstrate that the specific choice of GP model—including novel heteroscedastic and classifier formulations—significantly impacts accuracy, and they provide a new utility function to automate this selection for complex biological models.

What Problem Does This Paper Address?

In fields like population genetics, we often have complex simulation models but no "likelihood function" to tell us how well parameters fit the data. Approximate Bayesian Computation (ABC) solves this by running thousands of simulations and keeping parameters that produce data close to our observations.

The catch? A single simulation can take hours. To fix this, researchers use Bayesian Optimization for Likelihood-Free Inference (BOLFI), which uses a Gaussian Process (GP) as a "surrogate" to predict the results of simulations without running them. However, standard GPs assume the "noise" in simulations is constant and Gaussian—assumptions that rarely hold for the discrepancy measures used in ABC. This paper investigates how to choose the right GP model to ensure ABC remains both fast and accurate.

Methods

The authors evaluate several GP formulations to see which best handles the quirks of ABC discrepancies:

  1. Standard GP Regression: The baseline approach, often paired with transformations (like log or sqrt) to handle non-negative discrepancy values.
  2. Input-Dependent (Heteroscedastic) GP: A novel extension where the variance of the simulation noise is itself modeled by a second GP. This accounts for the fact that simulations might be "noisier" in certain parts of the parameter space.
  3. Classifier GP: Instead of modeling the exact discrepancy value, this models the probability that a simulation will fall below the ABC threshold directly using binary classification.
  4. Automated Model Selection: They introduce a "classifier utility" function. Unlike standard metrics that care about the whole model fit, this utility specifically measures how well a GP identifies whether a parameter set is "good" (below the ABC threshold) or "bad."

Datasets

The researchers validated their approach using:

  • 1D and 2D Toy Benchmarks: Including Gaussian, Poisson, Bimodal, and Uniform models, as well as the classic Lotka-Volterra predator-prey model.
  • Bacterial Genomic Data: A real-world dataset of 616 Streptococcus pneumoniae genomes. They focused on estimating rates of gene deletion, novel gene introduction, and horizontal gene transfer (HGT).

Key Insights and Ideas

  • The "Threshold" is Everything: In ABC, we only care if a simulation is "close enough" to the data. Therefore, a GP that fits the entire range of discrepancies perfectly is less useful than one that is highly accurate near the ABC threshold.
  • Transformations are Power Tools: Simply taking the square root of the discrepancy often made a standard GP perform as well as more complex heteroscedastic models by stabilizing the variance.
  • Heteroscedasticity Matters: Simulation noise often changes depending on the parameters. Ignoring this (as standard GPs do) leads to over- or under-confident posterior distributions.
  • Classifier Utility vs. MLPD: Standard model selection (MLPD) often picks suboptimal models for ABC because it prioritizes the fit of large, irrelevant discrepancy values. The new classifier utility consistently picks models that produce better posterior estimates.

Key Results

  • Accuracy Gains: In toy examples, the best-chosen GP models reduced the Total Variation (TV) distance (a measure of error) significantly compared to the baseline rejection sampler, especially when simulation budgets were low (e.g., only 50–200 evaluations).
  • Bacterial Insights: Applying their method to S. pneumoniae, they found that gene acquisitions significantly outpace deletions, suggesting a mechanism (like natural selection) must be counteracting genome growth.
  • HGT Origins: They estimated that the ratio of gene acquisitions from external sources versus internal population transfer is approximately 0.4, providing a quantitative look at how bacteria "trade" DNA.

Limitations and Future Work

  • Fixed Summary Statistics: The paper assumes the summary statistics and discrepancy measures are already chosen. Future work could integrate GP modeling with the automated selection of these statistics.
  • Search Space: The authors assumed the region of high posterior probability was known in advance. In a fully "blind" scenario, an initial exploration phase would be necessary.
  • Computational Overhead: While GPs save simulation time, fitting complex input-dependent GPs adds its own (smaller) computational cost, which could be a factor for very high-dimensional parameter spaces.

Summary updated

Abstract

Approximate Bayesian computation (ABC) can be used for model fitting when the likelihood function is intractable but simulating from the model is feasible. However, even a single evaluation of a complex model may take several hours, limiting the number of model evaluations available. Modelling the discrepancy between the simulated and observed data using a Gaussian process (GP) can be used to reduce the number of model evaluations required by ABC, but the sensitivity of this approach to a specific GP formulation has not yet been thoroughly investigated. We begin with a comprehensive empirical evaluation of using GPs in ABC, including various transformations of the discrepancies and two novel GP formulations. Our results indicate the choice of GP may significantly affect the accuracy of the estimated posterior distribution. Selection of an appropriate GP model is thus important. We formulate expected utility to measure the accuracy of classifying discrepancies below or above the ABC threshold, and show that it can be used to automate the GP model selection step. Finally, based on the understanding gained with toy examples, we fit a population genetic model for bacteria, providing insight into horizontal gene transfer events within the population and from external origins.
BibTeX
@article{jarvenpaa2018gaussian,
  author = {Marko Järvenpää and Michael U. Gutmann and Aki Vehtari and Pekka Marttinen},
  title = {{Gaussian process modelling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria}},
  journal = {The Annals of Applied Statistics},
  year = {2018},
  doi = {10.1214/18-aoas1150},
}
Figure