Score-Based Composition for Simulation-Based Inference

This direction is merged into another direction: likelihood-free inference (intractable likelihood functions). Its last version stays readable and citable here.

Version 1 of 17 papers16 conceptsfrozen 2026-10-04

In Simulation-Based Inference (SBI), we aim to recover the posterior distribution of parameters $\theta$ given some observed data $x$, using only a simulator that can generate synthetic data. Traditionally, this requires simulating large datasets that match the structure of our real-world observations. However, when we have many independent observations—often called "tall data"—or complex hierarchical structures, the computational cost of simulating these massive datasets becomes prohibitive.

A recent line of research, spearheaded by Tomas Geffner, George Papamakarios, and Andriy Mnih, addresses this by shifting the focus from estimating probability densities to estimating scores—the gradients of the log-posterior with respect to the parameters.

The Additive Property of Scores

The core intuition behind this approach is a simple property of logarithms. For a set of $n$ independent and identically distributed (i.i.d.) observations $x_{1:n} = \{x_1, \dots, x_n\}$, the joint posterior $p(\theta | x_{1:n})$ is proportional to the product of the individual likelihoods and the prior:

$$p(\theta | x_{1:n}) \propto p(\theta) \prod_{i=1}^n p(x_i | \theta)$$

If we take the gradient of the log-posterior (the "score"), the product turns into a sum:

$$\nabla_\theta \log p(\theta | x_{1:n}) = \nabla_\theta \log p(\theta) + \sum_{i=1}^n \nabla_\theta \log p(x_i | \theta)$$

As established in Score Modeling for Simulation-based Inference, this additivity allows us to decompose a complex inference problem into simple, modular pieces. Instead of training a model to understand a dataset of size $n$, we train a neural network to estimate the score for a single observation. At inference time, we simply sum the scores predicted for each individual data point in our real dataset.

Factorized Neural Posterior Score Estimation

This mechanism is formalized as Factorized Neural Posterior Score Estimation (F-NPSE). In Compositional Score Modeling for Simulation-Based Inference, the authors demonstrate that F-NPSE is highly sample-efficient. Because the model only needs to learn the relationship between $\theta$ and a single $x_i$, it requires far fewer simulator calls during training than methods that must see entire batches of data.

Once the score model is trained, we can perform "zero-shot" generalization to any number of observations. Whether we have 10 or 10,000 data points, we use the same pre-trained model and aggregate the results. This avoids the "simulation bottleneck" where models must be retrained or re-simulated whenever the volume of data changes.

Scaling to Massive Hierarchical Models

The power of composition extends beyond simple i.i.d. data. In Compositional amortized inference for large-scale hierarchical Bayesian models, Jonas Arruda, Jan Hasenauer, and colleagues apply this "divide-and-conquer" strategy to hierarchical models.

In a hierarchical setting, parameters might be shared across different groups of data. By training generative models on individual components or groups rather than the entire nested structure, they successfully scaled SBI to systems with hundreds of thousands of parameters. This approach, further detailed in Compositional amortized inference for large-scale hierarchical Bayesian models (ICLR 2026), transforms what would typically be hours of supercomputer time into minutes on a single GPU by using an error-damping estimator to stabilize the aggregated scores.

Stability and Speed in Tall Data

While summing scores is mathematically elegant, it introduces practical challenges. If the individual score estimates have small errors, do these errors explode when we sum thousands of them?

Camille Touron and Pedro L. C. Rodrigues provided the first theoretical grounding for this in Error analysis of a compositional score-based algorithm for simulation-based inference. They proved that the total error in the combined posterior remains bounded, justifying the use of compositional methods for large-scale scientific tasks.

To make the sampling process faster, Gabriel Cardoso and Julia Linhart introduced the GAUSS algorithm in Diffusion posterior sampling for simulation-based inference in tall data settings. By approximating the diffusion process of the combined posterior directly—rather than relying on slower iterative sampling like Langevin dynamics—they achieved significant speedups and better stability when dealing with "tall" datasets.

A New Engine for Scientific Inference

As summarized in the tutorial review Diffusion Models in Simulation-Based Inference by Stefan T. Radev and others, score-based modeling is becoming the standard "engine" for flexible SBI. By treating the posterior as a composition of modular score functions, researchers can:

  1. Reuse models: Train on single observations and apply to any dataset size.
  2. Save computation: Drastically reduce the number of simulator runs needed.
  3. Adapt flexibly: Change priors or data structures at inference time without retraining the core score model.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_score_based_composition_for_simulation_b_bfc876b4,
  title = {Score-Based Composition for Simulation-Based Inference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/score-based-composition-for-simulation-based-inference/rdr_bfc876b460d44e5cb73dff26a86ab925},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_score_based_composition_for_simulation_b_bfc876b4_v1,
  title = {Score-Based Composition for Simulation-Based Inference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/score-based-composition-for-simulation-based-inference/rdr_bfc876b460d44e5cb73dff26a86ab925/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}