sbi: A toolkit for simulation-based inference

Álvaro Tejero-Cantero, Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pedro J. Goncalves, David S. Greenberg, Jakob H. MackeJ. Open Source Softw 2020 · DOI · DBLP

Abstract

The sbi toolkit is a high-performance PyTorch-based library designed to facilitate state-of-the-art simulation-based inference (SBI) for the scientific community. By providing a unified interface for Bayesian inference on "black-box" simulators where likelihood functions are unavailable or computationally intractable, sbi enables researchers to perform complex statistical modeling with greater efficiency and accuracy than traditional methods. The toolkit implements several neural inference algorithms, including SNPE, SNLE, and SNRE, and is designed for both ease of use and high-performance scalability.

Introduction: The Challenge of Likelihood-Free Inference

In many scientific fields—ranging from neuroscience and systems biology to astrophysics and macroeconomics—researchers utilize complex stochastic simulators to model real-world phenomena. These simulators are essential for testing hypotheses and making predictions. However, they present a significant statistical hurdle: they are often "likelihood-free."

In traditional Bayesian inference, a likelihood function (the probability of observing specific data given a set of parameters) is required to calculate a posterior distribution (the probability of parameters given the observed data). For many modern simulators, the internal logic is so complex or involves so many latent variables that the likelihood is either impossible to derive analytically or too computationally expensive to evaluate.

Historically, researchers have turned to Approximate Bayesian Computation (ABC). ABC methods work by running millions of simulations and "rejecting" those that do not closely match the observed data. While conceptually simple, ABC is notoriously inefficient, often requiring an impractical number of simulations to achieve high precision, especially in high-dimensional parameter spaces. The sbi toolkit addresses these limitations by employing deep learning to learn the underlying statistical relationships between parameters and data, making inference significantly faster and more accurate.

The sbi Toolkit: Methods and Implementation

The sbi toolkit unifies several state-of-the-art neural inference algorithms under a single, user-friendly API. It leverages deep neural networks—specifically normalizing flows via the nflows package—to approximate the required probability distributions. Normalizing flows allow for the construction of complex, flexible distributions by transforming a simple base distribution through a series of trainable, invertible mappings.

The library implements three primary families of algorithms, each suited to different inference needs:

  1. Sequential Neural Posterior Estimation (SNPE): This method directly learns the posterior distribution. By training a neural network to map observed data to a distribution over parameters, SNPE provides an immediate estimate of which parameters are likely given the data.
  2. Sequential Neural Likelihood Estimation (SNLE): Instead of the posterior, this approach learns an emulator of the simulator's likelihood function. Once this surrogate model is trained, researchers can utilize standard statistical tools like Markov Chain Monte Carlo (MCMC) or Slice Sampling to explore and sample the posterior.
  3. Sequential Neural Ratio Estimation (SNRE): This method trains a classifier to distinguish between "correct" parameter-data pairs (drawn from the joint distribution) and "incorrect" ones (drawn from the product of marginals). This effectively estimates the likelihood ratio, which can then be used for posterior sampling.

A key innovation within sbi is the inclusion of trainable summarizing networks. In many applications, simulator output is high-dimensional (e.g., time series or images). These networks automatically learn to extract the most relevant features from raw data during the training process, significantly reducing the need for manual feature engineering and expert-defined summary statistics.

Design Principles and Key Features

The architecture of sbi is built around several key design choices that enhance its utility for both novice and expert researchers:

  • The NeuralPosterior Object: A central feature of the toolkit is the NeuralPosterior object. It wraps the trained neural network so that it behaves exactly like a standard PyTorch probability distribution. This allows users to sample from the posterior or evaluate log-probabilities using a familiar and consistent syntax, facilitating integration with other PyTorch-based tools.
  • Amortization vs. Sequential Inference: The toolkit supports "amortized" inference, where a model is trained once to work for any future observation within the prior range. For cases requiring extreme precision on a single experimental observation, sbi supports "sequential" inference, which refines the model over multiple rounds of simulation and training focused on the specific data of interest.
  • Handling "Broken" Simulations: Real-world simulators frequently encounter numerical instabilities or return invalid results (e.g., NaNs) for certain parameter ranges. sbi is engineered to handle these failed simulations gracefully, ensuring that the training pipeline remains robust and does not crash.
  • Scalability and Performance: By utilizing PyTorch, the toolkit can leverage GPU acceleration for neural network training. Furthermore, it integrates joblib to easily parallelize the running of simulations across multiple CPU cores, which is often the primary bottleneck in SBI workflows.

Software Maturity and Community Impact

As a software-focused contribution published in the Journal of Open Source Software, sbi represents the culmination and merging of functionality from several previous research packages, including delfi and lfi. It provides a stable, unified foundation for the growing SBI community.

The toolkit is designed with a "simple interface" philosophy, where a single function call with sensible defaults can perform complex inference. This significantly lowers the barrier to entry for scientists who are not machine learning experts. Simultaneously, the library remains highly modular, allowing power users to swap out neural network architectures, density estimators, or optimization routines while keeping the rest of the inference pipeline intact.

Limitations and Future Directions

While sbi significantly advances the accessibility of advanced inference techniques, it is not a "silver bullet." The quality of the resulting inference remains dependent on the fidelity of the simulator and the appropriateness of the chosen prior. Additionally, while the simple interface handles many cases, extremely complex or high-dimensional problems may still require careful hyperparameter tuning of the underlying neural networks.

The authors have established sbi as a community-oriented, open-source project. They actively invite contributions from the broader machine learning and scientific communities to incorporate new algorithms and features, ensuring that the toolkit remains at the cutting edge of simulation-based inference.

Summary updated

BibTeX
@article{tejerocantero2020sbi,
  author = {Álvaro Tejero-Cantero and Jan Boelts and Michael Deistler and Jan-Matthis Lueckmann and Conor Durkan and Pedro J. Goncalves and David S. Greenberg and Jakob H. Macke},
  title = {{sbi: A toolkit for simulation-based inference}},
  journal = {J. Open Source Softw.},
  year = {2020},
  doi = {10.21105/joss.02505},
}
Figure