swyft: Truncated Marginal Neural Ratio Estimation in Python

Benjamin Kurt Miller, Alex Cole, Christoph Weniger, Francesco Nattino, Ou Ku, Meiert W. GrootesJ. Open Source Softw 2022 · DOI · DBLP

Introduction

In fields ranging from astrophysics to epidemiology, complex numerical simulators are the primary tools for modeling physical phenomena. However, performing Bayesian inference—the process of determining the probability of model parameters given observed data—remains a significant computational challenge. Traditional methods like Markov chain Monte-Carlo (MCMC) are often prohibitively slow, requiring thousands of sequential simulator calls. Furthermore, many modern simulators are "likelihood-free," meaning they can generate synthetic data but do not provide an explicit mathematical formula for the probability of that data.

The swyft Python package addresses these bottlenecks by implementing Truncated Marginal Neural Ratio Estimation (TMNRE). This open-source framework enables high-performance simulation-based inference (SBI) by leveraging neural networks to learn the relationship between parameters and data, significantly reducing the number of simulations required for accurate posterior estimation.

Methodology: Truncated Marginal Neural Ratio Estimation (TMNRE)

The core innovation of swyft lies in its three-pronged approach to inference: Neural Ratio Estimation, Marginalization, and Truncation.

Neural Ratio Estimation (NRE)

Instead of attempting to calculate a likelihood directly, swyft trains a neural network (implemented in PyTorch) to estimate the ratio between the likelihood and the evidence. This learned ratio serves as a proxy for the posterior distribution, allowing for efficient inference without an explicit likelihood function.

Marginal Estimation

A common challenge in high-dimensional inference is the "curse of dimensionality." Often, researchers are only interested in a small subset of parameters, while dozens of others are "nuisance parameters." swyft circumvents this by estimating marginal posteriors directly. By focusing only on the parameters of interest, the package achieves much higher computational efficiency than methods that attempt to map the entire joint parameter space.

Prior Truncation

The "truncation" aspect of TMNRE is a sequential strategy designed to minimize wasted computation. swyft begins with coarse simulations to identify plausible regions of the parameter space. It then "truncates" the prior, focusing subsequent, more computationally expensive simulations only on these relevant regions. This iterative refinement ensures that the simulator is only called where it matters most.

Infrastructure and Scalability

swyft is designed for the era of big data. It integrates with Dask for distributed computing, allowing it to scale from a single laptop to large high-performance computing (HPC) clusters. Additionally, it utilizes Zarr for efficient data storage, which enables the seamless reuse of simulations across different runs and prevents redundant computational work.

Scientific Applications

The package has already demonstrated its utility across several complex scientific domains:

  • Strongly Lensed Galaxies: Inferring dark matter substructure from astronomical images.
  • Cosmic Microwave Background (CMB): Estimating cosmological parameters from simulated data.
  • Gravitational Waves: Accelerating the analysis of LIGO-Virgo triggers, where real-time response is critical.
  • Binary Neutron Star Mergers: Characterizing magnetohydrodynamics in scenarios where traditional likelihood-based methods are impossible.

Conclusion and Future Directions

swyft provides a flexible, "plug-and-play" tool for researchers across disciplines. By treating simulation time as the most expensive resource, it optimizes every stage of the inference pipeline. While future work aims to further automate prior truncation and handle even higher-dimensional nuisance spaces, swyft currently stands as a powerful solution for the next generation of simulation-based science.

Summary updated

BibTeX
@article{miller2022swyft,
  author = {Benjamin Kurt Miller and Alex Cole and Christoph Weniger and Francesco Nattino and Ou Ku and Meiert W. Grootes},
  title = {{swyft: Truncated Marginal Neural Ratio Estimation in Python}},
  journal = {J. Open Source Softw.},
  year = {2022},
  doi = {10.21105/joss.04205},
}
Figure