TL;DR
This paper improves the efficiency of Approximate Bayesian Computation (ABC) by using Gaussian Processes (GPs) to surrogate expensive simulations. The authors demonstrate that the specific choice of GP model—including novel heteroscedastic and classifier formulations—significantly impacts accuracy, and they provide a new utility function to automate this selection for complex biological models.
What Problem Does This Paper Address?
In fields like population genetics, we often have complex simulation models but no "likelihood function" to tell us how well parameters fit the data. Approximate Bayesian Computation (ABC) solves this by running thousands of simulations and keeping parameters that produce data close to our observations.
The catch? A single simulation can take hours. To fix this, researchers use Bayesian Optimization for Likelihood-Free Inference (BOLFI), which uses a Gaussian Process (GP) as a "surrogate" to predict the results of simulations without running them. However, standard GPs assume the "noise" in simulations is constant and Gaussian—assumptions that rarely hold for the discrepancy measures used in ABC. This paper investigates how to choose the right GP model to ensure ABC remains both fast and accurate.
Methods
The authors evaluate several GP formulations to see which best handles the quirks of ABC discrepancies:
- Standard GP Regression: The baseline approach, often paired with transformations (like
logorsqrt) to handle non-negative discrepancy values. - Input-Dependent (Heteroscedastic) GP: A novel extension where the variance of the simulation noise is itself modeled by a second GP. This accounts for the fact that simulations might be "noisier" in certain parts of the parameter space.
- Classifier GP: Instead of modeling the exact discrepancy value, this models the probability that a simulation will fall below the ABC threshold directly using binary classification.
- Automated Model Selection: They introduce a "classifier utility" function. Unlike standard metrics that care about the whole model fit, this utility specifically measures how well a GP identifies whether a parameter set is "good" (below the ABC threshold) or "bad."
Datasets
The researchers validated their approach using:
- 1D and 2D Toy Benchmarks: Including Gaussian, Poisson, Bimodal, and Uniform models, as well as the classic Lotka-Volterra predator-prey model.
- Bacterial Genomic Data: A real-world dataset of 616 Streptococcus pneumoniae genomes. They focused on estimating rates of gene deletion, novel gene introduction, and horizontal gene transfer (HGT).
Key Insights and Ideas
- The "Threshold" is Everything: In ABC, we only care if a simulation is "close enough" to the data. Therefore, a GP that fits the entire range of discrepancies perfectly is less useful than one that is highly accurate near the ABC threshold.
- Transformations are Power Tools: Simply taking the square root of the discrepancy often made a standard GP perform as well as more complex heteroscedastic models by stabilizing the variance.
- Heteroscedasticity Matters: Simulation noise often changes depending on the parameters. Ignoring this (as standard GPs do) leads to over- or under-confident posterior distributions.
- Classifier Utility vs. MLPD: Standard model selection (MLPD) often picks suboptimal models for ABC because it prioritizes the fit of large, irrelevant discrepancy values. The new classifier utility consistently picks models that produce better posterior estimates.
Key Results
- Accuracy Gains: In toy examples, the best-chosen GP models reduced the Total Variation (TV) distance (a measure of error) significantly compared to the baseline rejection sampler, especially when simulation budgets were low (e.g., only 50–200 evaluations).
- Bacterial Insights: Applying their method to S. pneumoniae, they found that gene acquisitions significantly outpace deletions, suggesting a mechanism (like natural selection) must be counteracting genome growth.
- HGT Origins: They estimated that the ratio of gene acquisitions from external sources versus internal population transfer is approximately 0.4, providing a quantitative look at how bacteria "trade" DNA.
Limitations and Future Work
- Fixed Summary Statistics: The paper assumes the summary statistics and discrepancy measures are already chosen. Future work could integrate GP modeling with the automated selection of these statistics.
- Search Space: The authors assumed the region of high posterior probability was known in advance. In a fully "blind" scenario, an initial exploration phase would be necessary.
- Computational Overhead: While GPs save simulation time, fitting complex input-dependent GPs adds its own (smaller) computational cost, which could be a factor for very high-dimensional parameter spaces.
Summary updated