Amortized Bayesian inference has revolutionized fields dealing with computationally expensive simulations, from gravitational-wave astronomy (Gabbard et al., Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy) to exoplanet atmospheric retrieval (Wildberger et al., Flow matching for atmospheric retrieval of exoplanets: Where reliability meets adaptive noise levels). By shifting the heavy computational burden to an offline training phase, neural networks can approximate complex posterior distributions $p(\theta|x)$ in milliseconds.
However, this speed comes with a hidden cost: prior fixation. Standard amortized models implicitly bake the training data distribution—and thus a specific prior $p(\theta)$—directly into the weights of the neural network (Legin et al., A Framework for Obtaining Accurate Posteriors of Strong Gravitational Lensing Parameters with Flexible Priors and Implicit Likelihoods using Density Estimation; Haldemann et al., Exoplanet characterization using conditional invertible neural networks). If a researcher later wants to inject new domain knowledge, enforce different physical boundaries, or perform a sensitivity analysis, the pre-trained model becomes invalid. To change the prior, one must completely retrain the network from scratch (Gloeckler et al., All-in-one simulation-based inference; Yang et al., PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference).
The Core Idea: Prior-Conditioned Networks
To break this rigidity, a new paradigm has emerged: prior-conditioned amortization. Instead of training a network on a single, fixed prior, the prior distribution itself is treated as a dynamic input variable at inference time.
The network no longer just learns to map data $x$ to a posterior. Instead, it learns a generalized mapping:
$$\text{Network}(x, \text{Prior}) \to p(\theta|x, \text{Prior})$$
By exposing the prior as an explicit runtime parameter, a single pre-trained model can instantly adapt to changing experimental conditions or alternative scientific hypotheses without requiring any retraining (Starostin et al., Fast and Reliable Probabilistic Reflectometry Inversion with Prior-Amortized Neural Posterior Estimation).
How to Feed a Distribution into a Neural Network
Representing an entire probability distribution as a network input requires creative architectural choices. Researchers have proposed several distinct mechanisms to achieve this:
1. Context Variables and Hyperparameters
One straightforward approach is to parameterize the prior (e.g., its bounds, mean, or variance) and pass these parameters as context tokens. For instance, Elsemüller et al. developed Sensitivity-Aware Amortized Bayesian Inference (Sensitivity-Aware Amortized Bayesian Inference), which feeds modeling choices directly into the network as context variables, enabling instant sensitivity analysis. Similarly, Zhang introduced the Any Parameter Encoder (Rapid Model Comparison by Amortizing Across Models) to generalize inference across varying model configurations and hyperparameter settings.
2. Binned Histograms
For arbitrary or non-parametric user beliefs, Chang et al. introduced the Amortized Conditioning Engine (Amortized Probabilistic Conditioning for Optimization, Simulation and Inference). The authors convert one-dimensional prior probability density functions into normalized histograms, which a learned projection maps directly into a Transformer's context set. This allows users to inject custom shapes of domain knowledge at runtime.
3. Distribution Tokens
Moving further into sequence models, Whittle et al. proposed Distribution Transformers (Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptation). They represent distributions as Gaussian Mixture Models (GMMs) and process them as sequences of tokens, allowing the network to perform recursive, on-the-fly prior adaptation akin to classical filtering but with deep learning speeds.
4. Test-Time Guidance
Rather than feeding the prior forward through the network architecture, some methods modify the generative process at test time. Yang et al. (PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference) leverage amortized diffusion models and adapt them during inference to accommodate new expert knowledge or hypotheses without retraining.
Why It Works: Learning the Meta-Bayesian Operator
To understand why this works intuitively, consider what a standard amortized network learns. It acts as a specialized calculator, optimized for one specific setup. A prior-conditioned network, by contrast, acts as a meta-learner.
During the offline training phase, the model is exposed to a wide variety of simulated priors—ranging from broad uninformative distributions to narrow, highly constrained ones. By training across this distribution of distributions (often leveraging architectures like Transformers or Invertible Neural Networks), the network learns the underlying mathematical relationship between the prior, the likelihood, and the resulting posterior.
When a new prior is injected at test time, the network uses its learned internal representations to shift, scale, or reshape the posterior distribution accordingly. As noted by Huang et al. (ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition) and Rakotoarison et al. ($\alpha$-PFN: Fast Entropy Search via In-Context Learning), this capability transforms amortized inference from a rigid, single-use tool into a flexible engine capable of active data acquisition, real-time optimization, and rapid hypothesis testing.
Key Foundational Work
Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy (2019) — contributed 1 concept elements to this direction.
A Framework for Obtaining Accurate Posteriors of Strong Gravitational Lensing Parameters with Flexible Priors and Implicit Likelihoods using Density Estimation (2022) — contributed 1 concept elements to this direction.
Rapid Model Comparison by Amortizing Across Models (2019) — contributed 1 concept elements to this direction.