Introduction
In the realm of simulation-based science, researchers rely on complex computer programs to model phenomena ranging from epidemiological outbreaks to cognitive processes. A significant hurdle in this field is the lack of a closed-form mathematical expression for the "likelihood"—the probability of observing specific data given a set of parameters. Traditionally, researchers have addressed this through two separate paths: Simulation-Based Inference (SBI) to estimate the posterior, and Surrogate Modeling (SM) to create fast emulators of the simulator. JANA (Jointly Amortized Neural Approximation) bridges this gap by learning both components simultaneously in a fully amortized framework.
The JANA Architecture
JANA employs an end-to-end neural architecture designed to handle the dual tasks of posterior estimation and likelihood emulation. The framework consists of three primary components:
- Summary Network ($H_\psi$): This network compresses high-dimensional or complex data (e.g., time series or images) into a lower-dimensional embedding vector.
- Posterior Network ($P_\phi$): Utilizing a conditional invertible neural network (cINN), this component learns the mapping from data embeddings to parameter space to approximate the posterior distribution $p(\theta|x)$.
- Likelihood Network ($L_\eta$): Another cINN that emulates the simulator, learning to approximate the likelihood $p(x|\theta)$.
The networks are trained using a joint objective function that minimizes the negative log-probabilities of both the likelihood and posterior. This joint training acts as a form of mutual regularization. Additionally, a Maximum Mean Discrepancy (MMD) penalty is applied to the summary space to ensure it follows a standard distribution, facilitating the detection of model misspecification when the system encounters data outside its training distribution.
Key Innovations and Advantages
JANA introduces several critical improvements over traditional, isolated methods:
- Full Amortization: Once trained, JANA can perform inference on new data in milliseconds, making it suitable for real-time applications and large-scale datasets.
- Joint Calibration (JSBC): By learning the likelihood and posterior together, JANA enables a new diagnostic tool where the consistency between the two networks can be checked. Discrepancies between the posterior network's predictions and the likelihood network's synthetic data signal potential training errors or model limitations.
- Efficient Model Comparison: Because JANA utilizes normalizing flows (cINNs), it can compute the marginal likelihood—a key metric for model comparison—using a direct algebraic formula, bypassing the need for expensive numerical integration.
Experimental Results
The framework was validated across various benchmarks, including standard SBI tasks and complex real-world models.
Disease Outbreak Modeling
JANA was applied to a complex 34-parameter Markovian compartmental model for disease outbreaks. The results demonstrate that the JANA surrogate can accurately capture not only the mean trajectory of the outbreak (Infected, Recovered, Dead) but also the inherent aleatoric uncertainty of the stochastic system.
Benchmark Performance
On the "Two Moons" benchmark, a classic bimodal problem, JANA achieved results comparable to or better than state-of-the-art sequential methods while maintaining the speed advantages of amortization. In decision-making models (Exchangeable Diffusion Model), JANA's marginal likelihood estimates showed a near-perfect correlation ($r = 1.000$) with traditional gold-standard methods.
Conclusion
JANA represents a significant step forward in simulation-based inference by unifying posterior estimation and surrogate modeling. While it relies on the "small world" assumption that the simulator is a faithful representation of reality, its built-in misspecification detection and joint calibration tools provide robust safeguards. Future work may explore weight sharing between components to further enhance efficiency and reduce parameter overhead.
Summary updated