POPE: post optimization posterior evaluation of likelihood free models

Edward Meeds, Michael Chiang, Mary Lee, Olivier Cinquin, John S. Lowengrub, Max Welling

Introduction

In fields such as biology and physics, complex simulators are often used to model natural phenomena. These simulators are frequently "likelihood-free," meaning that while they can produce outputs for given inputs, the underlying probability density function (the likelihood) is mathematically intractable. Standard practice involves "blind optimization"—finding a single parameter set ($\theta^*$) that minimizes an objective function. However, a point estimate fails to capture parameter sensitivities, correlations, or the impact of biological constraints.

Post-Optimization Posterior Evaluation (POPE) is a Bayesian framework designed to look "under the hood" of these models after an optimal parameter set has been identified. By leveraging Approximate Bayesian Computation (ABC) with novel one-sided kernels, POPE allows researchers to visualize the optimization landscape and assess the robustness of their findings.

Methodology

POPE builds upon the foundation of ABC. In standard ABC, a simulation is accepted if its output $y_{sim}$ is sufficiently close to an observation $y_{obs}$. POPE modifies this approach by focusing on the optimization threshold: it checks if $y_{sim} \leq y^*$, where $y^*$ is the previously found optimal value.

One-Sided Kernels and Adaptive Sampling

To avoid the limitations of hard boundaries in MCMC sampling, POPE introduces "soft" one-sided kernels (e.g., Gaussian or Exponential). These kernels assign a likelihood of 1 to all values better than the threshold and apply a smooth penalty to those that are worse.

Furthermore, the framework includes an Adaptive POPE algorithm. This online method adaptively updates the tolerance ($\epsilon$) and the objective threshold ($y^*$) during sampling, which is particularly effective for denoising stochastic simulators where the true optimum might be obscured by noise.

Case Studies and Results

The authors evaluated POPE using two distinct biological models: a stochastic stem-cell niche geometry model and a deterministic reaction-diffusion model for colon cancer patterns.

Stem-Cell Niche Geometry

Modeling the physical shape of a cell "niche" in C. elegans revealed that constraints are as critical as the primary objective. Without realistic geometry constraints, the model favored unrealistic "jumps" in cell row sizes to minimize production time. POPE demonstrated that applying these constraints shifted the posterior toward more biologically plausible shapes.

Comparison of niche geometry posteriors and predictive distributions for different tau values.

Colon Cancer Patterning

In a reaction-diffusion model simulating metabolic patterns in tumors, POPE identified three distinct "modes" (spotting patterns) that satisfied the optimization criteria. While a standard optimizer would likely find only one, POPE quantified the probability of each mode, offering a richer understanding of the model's capabilities and validating that certain decay rates are primary drivers of spot formation.

Discussion and Key Insights

A central insight of POPE is that constraints function as likelihoods. Biological constraints often shape the posterior distribution as significantly as the main objective function. By mapping the "neighborhood" around an optimum, POPE reveals whether a solution sits on a fragile spike or a robust plateau.

While POPE provides "free" sensitivity and correlation analysis through visualization, it is a post-optimization tool and not a replacement for global search algorithms. Future work involves integrating POPE with surrogate models, such as Gaussian Processes, to handle high-fidelity simulators that are computationally expensive to run thousands of times.

Summary updated

Abstract

In many domains, scientists build complex simulators of natural phenomena that encode their hypotheses about the underlying processes. These simulators can be deterministic or stochastic, fast or slow, constrained or unconstrained, and so on. Optimizing the simulators with respect to a set of parameter values is common practice, resulting in a single parameter setting that minimizes an objective subject to constraints. We propose a post optimization posterior analysis that computes and visualizes all the models that can generate equally good or better simulation results, subject to constraints. These optimization posteriors are desirable for a number of reasons among which easy interpretability, automatic parameter sensitivity and correlation analysis and posterior predictive analysis. We develop a new sampling framework based on approximate Bayesian computation (ABC) with one-sided kernels. In collaboration with two groups of scientists we applied POPE to two important biological simulators: a fast and stochastic simulator of stem-cell cycling and a slow and deterministic simulator of tumor growth patterns.
BibTeX
@article{meeds2015pope,
  author = {Edward Meeds and Michael Chiang and Mary Lee and Olivier Cinquin and John S. Lowengrub and Max Welling},
  title = {{POPE: post optimization posterior evaluation of likelihood free models}},
  journal = {BMC Bioinformatics},
  year = {2015},
  doi = {10.1186/s12859-015-0658-1},
}
Figure