Abstract
The dataset accompanying Truncated Marginal Neural Ratio Estimation. The software repository can be found here. The arxiv article can be found here.
Introduction
Bayesian inference is the gold standard for scientific parameter estimation, but it often hits a wall when dealing with "black-box" simulators where the likelihood is unknown. In many scientific fields—from cosmology to epidemiology—researchers use complex stochastic simulators to model the world. Performing Bayesian inference with these simulators is hard because they often have many parameters and an "intractable likelihood."
Existing Simulation-Based Inference (SBI) methods typically fall into two categories: sequential methods, which are simulation-efficient but difficult to validate, and amortized methods, which are easy to test but prohibitively expensive for high-dimensional problems. Furthermore, scientists usually only care about 1D or 2D marginal posteriors, yet most algorithms struggle by trying to solve the much harder problem of the full joint posterior first.
Truncated Marginal Neural Ratio Estimation (TMNRE)
TMNRE is a simulation-based inference algorithm that achieves high efficiency by directly targeting low-dimensional marginal posteriors and iteratively "zooming in" on relevant parameter regions. It combines three powerful ideas:
- Marginal Neural Ratio Estimation (MNRE): Instead of learning the full joint posterior, TMNRE uses neural networks as classifiers to directly estimate the likelihood-to-evidence ratio for 1D and 2D marginals, significantly reducing task complexity.
- Iterative Prior Truncation: The algorithm runs in rounds. In each round, it identifies regions of the parameter space with negligible posterior density and "truncates" them. This focuses subsequent simulations on the most plausible parameter values.
- Local Amortization: By learning the posterior over a constrained region rather than a single point, TMNRE enables "local amortization." This allows practitioners to perform empirical "coverage tests" to ensure their results are trustworthy.
Experimental Validation
The authors validated TMNRE across a diverse set of challenges, including the standard SBI Benchmark (e.g., Two Moons, SLCP, Lotka-Volterra), a 3-dim Torus Model with a narrow non-linear posterior, a highly multimodal 10-dim Eggbox problem, and a real-world Cosmological Simulator involving Cosmic Microwave Background (CMB) power spectra.
Key Results
TMNRE demonstrated top-tier performance on the standard SBI benchmark, outperforming traditional methods like ABC and proving competitive with or superior to state-of-the-art neural methods, particularly on tasks with narrow posteriors.
Scientific insight is almost always based on marginalization. By targeting marginals directly, TMNRE avoids the "curse of dimensionality" that plagues joint posterior estimation. In early rounds with limited data, the algorithm naturally produces wider, "conservative" posteriors, ensuring that the truncation process doesn't accidentally discard true parameter values. In the cosmology task, TMNRE produced accurate results with only 5,000 simulations—a fraction of what standard MCMC methods would require.
Limitations and Future Work
Current limitations include the use of axis-aligned "hyperrectangles" for truncation, which is less efficient for highly correlated parameters. Additionally, because it targets marginals, TMNRE does not provide the full joint posterior. Future directions include exploring more flexible truncation shapes (like hyperellipses) and integrating ideas from nested sampling to further improve efficiency in highly correlated spaces.
Summary updated