Gaussian process modeling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria

Marko Järvenpää, Michael U. Gutmann, Aki Vehtari, Pekka Marttinen

Abstract

Approximate Bayesian computation (ABC) can be used for model fitting when the likelihood function is intractable but simulating from the model is feasible. However, even a single evaluation of a complex model may take several hours, limiting the number of model evaluations available. Modelling the discrepancy between the simulated and observed data using a Gaussian process (GP) can be used to reduce the number of model evaluations required by ABC, but the sensitivity of this approach to a specific GP formulation has not yet been thoroughly investigated. We begin with a comprehensive empirical evaluation of using GPs in ABC, including various transformations of the discrepancies and two novel GP formulations. Our results indicate the choice of GP may significantly affect the accuracy of the estimated posterior distribution. Selection of an appropriate GP model is thus important. We formulate expected utility to measure the accuracy of classifying discrepancies below or above the ABC threshold, and show that it can be used to automate the GP model selection step. Finally, based on the understanding gained with toy examples, we fit a population genetic model for bacteria, providing insight into horizontal gene transfer events within the population and from external origins.

Introduction

In fields like population genetics, complex simulation models often lack an explicit likelihood function, making traditional Bayesian inference difficult. Approximate Bayesian Computation (ABC) addresses this by comparing simulated data to observations via a discrepancy measure. However, when simulations are computationally expensive—taking hours for a single run—standard ABC becomes infeasible. Bayesian Optimization for Likelihood-Free Inference (BOLFI) mitigates this by using a Gaussian Process (GP) as a surrogate model for the discrepancy. This study investigates the sensitivity of BOLFI to the specific GP formulation, a critical but previously under-explored factor in inference accuracy.

Methods

The authors evaluate several GP formulations to handle the non-Gaussian and heteroscedastic nature of ABC discrepancies:

  1. Standard GP Regression: Often used with log or sqrt transformations to enforce non-negativity.
  2. Input-Dependent (Heteroscedastic) GP: A novel approach where a second GP models the variance of simulation noise, accounting for parameter-dependent stochasticity.
  3. Classifier GP: Models the probability of a simulation falling below the ABC threshold directly.
  4. Automated Model Selection: A new "classifier utility" function is introduced to select the best GP model based on its ability to distinguish "good" parameters (those below the ABC threshold) from "bad" ones.
Comparison of GP formulations and transformations on discrepancy modeling.

The methodology was validated on 1D and 2D toy benchmarks (e.g., Gaussian, Poisson, Lotka-Volterra) and a real-world dataset of 616 Streptococcus pneumoniae genomes to estimate horizontal gene transfer (HGT) rates.

Results and Insights

The choice of GP model significantly impacts the accuracy of the estimated posterior. A key finding is that the GP's performance near the ABC threshold is more important than its global fit. Simple transformations, such as the square root of the discrepancy, can often stabilize variance enough for standard GPs to perform competitively with more complex models. However, ignoring heteroscedasticity when it is present can lead to poorly calibrated posteriors.

True vs. estimated posterior distributions across various test problems, showing KL and TV metrics.

In toy examples, optimized GP models significantly reduced the Total Variation (TV) distance compared to baseline rejection samplers under low simulation budgets. Applied to S. pneumoniae, the model revealed that gene acquisitions outpace deletions and estimated the ratio of external to internal gene acquisitions at approximately 0.4, providing quantitative evidence for HGT dynamics.

Conclusion

While GP surrogates drastically reduce simulation requirements, the choice of model is paramount. The proposed classifier utility provides a robust way to automate this selection. Future work could extend this framework to include automated selection of summary statistics and explore high-dimensional parameter spaces where initial exploration phases may be required.

Summary updated

Abstract

Approximate Bayesian computation (ABC) can be used for model fitting when the likelihood function is intractable but simulating from the model is feasible. However, even a single evaluation of a complex model may take several hours, limiting the number of model evaluations available. Modelling the discrepancy between the simulated and observed data using a Gaussian process (GP) can be used to reduce the number of model evaluations required by ABC, but the sensitivity of this approach to a specific GP formulation has not yet been thoroughly investigated. We begin with a comprehensive empirical evaluation of using GPs in ABC, including various transformations of the discrepancies and two novel GP formulations. Our results indicate the choice of GP may significantly affect the accuracy of the estimated posterior distribution. Selection of an appropriate GP model is thus important. We formulate expected utility to measure the accuracy of classifying discrepancies below or above the ABC threshold, and show that it can be used to automate the GP model selection step. Finally, based on the understanding gained with toy examples, we fit a population genetic model for bacteria, providing insight into horizontal gene transfer events within the population and from external origins.
BibTeX
@article{jarvenpaa2018gaussian,
  author = {Marko Järvenpää and Michael U. Gutmann and Aki Vehtari and Pekka Marttinen},
  title = {{Gaussian process modeling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria}},
  journal = {The Annals of Applied Statistics},
  year = {2018},
  doi = {10.1214/18-aoas1150},
}
Figure