Deep Generative Quantile Bayes

Jungeum Kim, Percy S. Zhai, Veronika RockováAISTATS 2025

TL;DR

This paper introduces Deep Generative Quantile Bayes, a novel method for sampling from complex Bayesian posteriors by learning multivariate quantile mappings. By leveraging optimal transport theory and neural networks, the authors create a "push-forward" sampler that can directly generate Bayesian credible sets and handle "implicit" models where the likelihood function is impossible to write down but easy to simulate.

What Problem Does This Paper Address?

In Bayesian statistics, we want to find the posterior distribution—the updated belief about parameters $\theta$ after seeing data $X$. Standard tools like Markov Chain Monte Carlo (MCMC) work well when we have a clear formula for the likelihood. However, many modern scientific models are "implicit": we can simulate data from them, but we don't have a tractable likelihood function.

Existing "likelihood-free" methods like Approximate Bayesian Computation (ABC) are often computationally expensive because they must be re-run from scratch for every new piece of data. Furthermore, defining "quantiles" (like the median or the 95th percentile) in multiple dimensions is mathematically ambiguous. This paper tackles these challenges by creating a reusable, deep-learning-based sampler that understands multivariate geometry.

Methods

The core innovation is the use of Vector Quantile Learning based on optimal transport.

  1. Brenier Maps: The authors define a multivariate quantile as the gradient of a convex potential function. This resolves the ambiguity of "ordering" data in high-dimensional space.
  2. Monge-Kantorovich (MK) Depth: They utilize MK depth to define "credible sets" (the Bayesian version of confidence intervals). By sampling from a unit ball in the latent space and pushing it through the learned map, they can directly visualize the shape of the posterior.
  3. Automatic Summary Statistics: To handle different types of data, they integrate specialized neural architectures:
  4. DeepSets for exchangeable (i.i.d.) data.
  5. LSTMs for time-series or dependent data.
  6. Training: The model is trained on triplets of $\{\theta, X, U\}$—parameters, simulated data, and random noise—using a dual-form objective function that ensures the mapping is a valid gradient of a convex function.

Datasets

The authors evaluate their method on two primary benchmarks:

  • Gaussian Conjugate Simulation: A standard "toy" model where the true posterior is known mathematically. This allows the authors to prove their method correctly exhibits "support shrinkage" (the posterior getting tighter as more data arrives).
  • Brock Hommes Model: A sophisticated economic agent-based model used to simulate asset trading in stock markets. This model features complex interactions and lacks a closed-form likelihood, making it a perfect test for likelihood-free inference.

Key Insights and Ideas

  • Support Shrinkage as a Feature: One of the most striking findings is that by using a "DeepSet" architecture to extract summary statistics, the model naturally learns to "shrink" its predictions as the sample size $n$ increases. This mimics the behavior of a true Bayesian posterior.
  • Direct Credible Set Sampling: Unlike MCMC, where you have to cluster points to find a 95% region, this method allows you to simply sample noise from a ball of radius 0.95. The neural network then maps that ball directly into the 95% credible set in parameter space.
  • Amortized Inference: Once the model is trained on simulations, it can be applied to any new real-world observation $X$ almost instantaneously. This "train once, use many times" approach is a massive speedup over traditional ABC.

Key Results

  • Accuracy: In the Brock Hommes model, the proposed method achieved a Distance to Measure (DTM) of 0.177, significantly outperforming Rejection ABC (0.521) and Sequential Monte Carlo ABC (0.513).
  • Efficiency: While training takes time (37 minutes in their test), the resulting sampler is "reusable." Traditional ABC methods would require 29–41 minutes per new observation, whereas this method processes new observations in milliseconds after the initial training.
  • Theoretical Soundness: The authors provide frequentist proofs showing that as the number of simulations $N$ goes to infinity, the estimated vector quantile converges to the true posterior.

Limitations and Future Work

  • Low-Dimensional Parameters: The current study focuses on models with relatively low-dimensional continuous parameter spaces. Scaling this to thousands of parameters (like in deep neural network weights) remains a challenge.
  • Continuous vs. Discrete: The framework is designed for continuous parameters; extending it to discrete parameter spaces (like classification categories) is noted as an area for future research.
  • Simulation Gap: The training relies on the simulation model being a good match for reality. If the observed data $X_0$ is very different from anything seen during training, the accuracy may drop. The authors suggest "importance weighting" as a potential fix for this in future iterations.

Summary updated

BibTeX
@inproceedings{kim2025deep,
  author = {Jungeum Kim and Percy S. Zhai and Veronika Rocková},
  title = {{Deep Generative Quantile Bayes}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
}
Figure