BlackBIRDS: Black-Box Inference foR Differentiable Simulators

Arnau Quera-Bofarull, Joel Dyer, Anisoara Calinescu, J. Doyne Farmer, Michael J. WooldridgeJ. Open Source Softw 2023 · DOI · DBLP

Introduction

BlackBIRDS is an open-source Python library designed to streamline parameter inference for differentiable simulators. By exploiting the gradient information inherent in PyTorch-based models, the library enables researchers to employ advanced Bayesian and frequentist inference techniques. These methods offer significant improvements in computational efficiency over traditional gradient-free Simulation-Based Inference (SBI) approaches.

The Challenge of Parameter Estimation

In scientific domains such as epidemiology and economics, complex simulations are essential for modeling real-world phenomena. A critical task in this process is parameter estimation—identifying the input values that allow a simulation to accurately replicate observed data.

Historically, many simulators have been treated as "black boxes," where the relationship between inputs and outputs is not analytically tractable. This necessitated the use of gradient-free SBI methods, which rely on repeated sampling and comparison. Such approaches are often prohibitively slow and computationally expensive. The recent emergence of differentiable simulators, built on frameworks like PyTorch, offers a solution by providing the gradient (sensitivity) of outputs with respect to inputs. BlackBIRDS was developed to provide a unified, user-friendly interface to leverage these gradients for diverse inference tasks.

Methodology and Framework

BlackBIRDS provides an object-oriented framework built on the PyTorch ecosystem, supporting three primary inference pathways:

  1. Simulated Minimum Distance (SMD): A frequentist method that identifies optimal parameter points by minimizing a specified loss function.
  2. Markov Chain Monte Carlo (MCMC): Specifically, the library implements Metropolis-adjusted Langevin Dynamics (MALA). Unlike standard MCMC, which utilizes random walks, MALA uses gradient information to guide the sampler toward regions of higher probability, drastically increasing sampling efficiency.
  3. Variational Inference (VI): By integrating with the normflows library, BlackBIRDS utilizes Normalizing Flows—neural networks capable of learning complex posterior distributions—to treat inference as an optimization problem.

The package is engineered for high-performance computing, featuring support for multi-GPU parallelization via the Message Passing Interface (MPI) and accommodating both forward and reverse-mode automatic differentiation.

Applications and Real-World Utility

As a software-focused contribution, BlackBIRDS has already demonstrated its utility in several scientific research contexts:

  • Epidemiology: Calibrating differentiable agent-based models to track and predict disease spread.
  • Economics: Modeling complex systems such as asset pricing.

The library's ability to reduce the number of required simulation runs through gradient-assisted efficiency not only accelerates research but also contributes to sustainability by lowering the financial and environmental costs associated with large-scale scientific computing.

Conclusion and Future Work

BlackBIRDS successfully lowers the barrier to entry for sophisticated inference algorithms by providing a "black-box" interface for differentiable models. While currently optimized for PyTorch, the framework's primary limitation is its requirement for differentiability, precluding its use with legacy non-differentiable codebases. Future development aims to expand the suite of supported algorithms, with Hamiltonian Monte Carlo (HMC) identified as a key addition to further enhance the library's Bayesian inference capabilities.

Summary updated

BibTeX
@article{querabofarull2023blackbirds,
  author = {Arnau Quera-Bofarull and Joel Dyer and Anisoara Calinescu and J. Doyne Farmer and Michael J. Wooldridge and Arnau Quera-Bofarull and Joel Dyer and J. Doyne Farmer and Ani Calinescu},
  title = {{BlackBIRDS: Black-Box Inference foR Differentiable Simulators}},
  journal = {J. Open Source Softw.},
  year = {2023},
  doi = {10.21105/joss.05776},
}
Figure