Compressed Simulation-Based Inference: Neural Posterior Estimation for Massive Data via Low-Dimensional Sketches

Michael Wieck-Sosa, Cosma Rohilla ShaliziarXiv 2026 · arXiv

TL;DR

This paper introduces a highly efficient method for Neural Posterior Estimation (NPE) that replaces complex, learned embedding networks with simple "randomized sketches" of data. By using a small number of random Fourier features ($2p+1$ for a $p$-dimensional parameter), the authors achieve state-of-the-art accuracy with significantly lower computational overhead, providing a scalable solution for simulation-based inference on massive datasets.

What Problem Does This Paper Address?

Simulation-Based Inference (SBI) is essential when we have a complex simulator but no tractable likelihood function. A popular approach is Neural Posterior Estimation (NPE), which trains a neural network to map observations directly to a posterior distribution.

However, NPE hits a wall with high-dimensional data (the "curse of dimensionality"). To fix this, researchers usually compress data into "summary statistics." Traditionally, this meant either:

  1. Hand-crafting summaries: Requires deep domain expertise and is often sub-optimal.
  2. Learning summaries: Using "embedding networks" (like CNNs or Transformers) to learn features. While accurate, training these networks on massive datasets is computationally expensive and slow.

The authors address this bottleneck by asking: Can we get the benefits of automated feature extraction without the heavy cost of training an embedding network?

Methods

The authors propose using randomized summary statistics, specifically low-dimensional sketches based on Random Fourier Features (RFF).

Instead of learning weights, they randomly draw $k$ functions ($\phi_1, \dots, \phi_k$) from a distribution. For time-series data, these functions are applied to subsequences (lags) and then time-averaged: $$S_n(X_{1:n}) = \frac{1}{n-m} \sum_{t=m+1}^{n} \phi(X_{t-m:t})$$

Key Innovations: * Theoretical Guarantee: They prove that for a model with a $p$-dimensional parameter, only $2p + 1$ random sketches are needed to ensure posterior consistency (meaning the estimate converges to the true value as data increases). * Efficiency: Because the "embedding" weights are random and fixed, there is no backpropagation through an embedding network. The only thing being trained is the density estimator (e.g., a Neural Spline Flow). * Architecture: This approach can be viewed as a 1D-CNN where the filters are random Fourier features and the pooling layer is a simple time-average.

Datasets

The method was evaluated on four distinct Data Generating Processes (DGPs) representing common challenges in time-series analysis:

  1. Moving Average (MA) Process: Uses the intractable g-and-k distribution for innovations.
  2. Autoregressive (AR) Process: Also uses g-and-k innovations, testing non-linear dependencies.
  3. State-Space Model (SSM): Features 25-dimensional observations, testing the method's ability to handle high-dimensional data.
  4. Regime-Switching Stochastic Volatility Model: A complex model that violates some of the paper's theoretical assumptions, testing the method's robustness.

Key Insights and Ideas

  • Agnostic Scaling: The sketching method is agnostic to the sample size ($n$) and the observation dimension ($d$). This makes it uniquely suited for "massive data" scenarios where Transformers or CNNs might struggle to fit in memory or take days to train.
  • Linear Complexity: The number of required sketches scales linearly with the number of parameters ($p$), not the complexity of the data itself.
  • Randomness is Enough: The most surprising insight is that optimized, learned embeddings are often unnecessary for NPE. Fixed random features capture enough information to match the performance of state-of-the-art architectures.

Key Results

The authors compared their "Sketching" approach against two heavyweights: Causal CNNs and Transformers.

  • Accuracy: In terms of Wasserstein-1 distance to the true parameter, the Sketching method matched or exceeded the performance of the Causal CNN across all DGPs. Comparison of posterior density quality across different sample sizes.

  • Computational Speed: The Sketching method was significantly faster. For the State-Space model, it reduced total computation time (preprocessing + training) by orders of magnitude compared to Transformers.

  • Optimization Speed: Because the summary statistics are pre-computed and low-dimensional, the Adam optimizer updates were much faster, leading to quicker convergence during the density estimation phase.
Figure 5. Comparison of posterior density quality. Wasserstein- 1 distance to δθ0 at different sample sizes, averaged over 1000 trajectories simulated under Pθ0,n.
Marginal posterior densities for the Moving Average process.

Limitations and Future Work

  • Assumptions: The theoretical proof relies on the model having a "real analytic" structure. While the experiments showed the method works even when this is violated (DGP 4), the theoretical bounds might not hold for all edge cases.
  • Data Types: The current work focuses on sequential/time-series data. The authors suggest that extending these randomized sketches to other data structures, such as graphs or spatiotemporal data, is a primary direction for future research.
  • Hyperparameter Selection: While the number of features $k$ is theoretically guided, users still need to choose the "lag" order ($m$) based on the model's transition dynamics.

Summary updated

Abstract

Neural posterior estimation (NPE) is a popular Bayesian simulation-based inference (SBI) approach that uses neural networks trained on simulated data--parameter pairs to approximate the posterior given a user-chosen prior. For high-dimensional observations, NPE methods typically approximate the conditional density of the parameter given a user-chosen or learned summary statistic, rather than the entire dataset, due to the curse of dimensionality. However, both approaches face challenges: hand-crafting good summaries for complex models requires considerable effort from scientists, while learning them from massive datasets can be prohibitively computationally expensive. To address these challenges, we propose approximating the conditional density of a parameter given low-dimensional sketches of the data. We show that, for a model with a $p$-dimensional parameter, only $2p+1$ random sketches are needed for posterior consistency. We compare with state-of-the-art embedding networks for NPE, and we find that our method \emph{matches} the best embedding network's performance while \emph{increasing computational efficiency}.
BibTeX
@misc{wiecksosa2026compressed,
  author = {Michael Wieck-Sosa and Cosma Rohilla Shalizi},
  title = {{Compressed Simulation-Based Inference: Neural Posterior Estimation for Massive Data via Low-Dimensional Sketches}},
  year = {2026},
  eprint = {2611.21286},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21286},
}
Figure