A hybrid two-level MCMC framework to accelerate posterior mean estimation with deep learning surrogates for bayesian inverse problems

Juntao Yang, Jeff Adie, Simon See, Adriano Gualandi, Gianmarco Mengaldo

Sure, here's a blog-style summary of the paper:

TL;DR

This paper introduces a novel "hybrid two-level MCMC" method that combines the speed of deep learning surrogates with the accuracy of high-fidelity numerical models to accelerate Bayesian inverse problems. This approach achieves the same accuracy as traditional methods but with a fraction of the computational cost, making complex simulations more accessible.

What Problem Does This Paper Address?

Bayesian inverse problems are crucial in many scientific fields, but solving them, especially those involving partial differential equations (PDEs), is computationally very expensive. This is because these problems often require repeatedly evaluating complex forward models. While deep learning (DL) surrogates can speed up these evaluations, they often lack the accuracy of high-fidelity numerical models, limiting their practical use. This paper addresses the challenge of achieving both speed and accuracy in Bayesian inverse problems by developing a method that leverages the strengths of both DL surrogates and traditional numerical solvers.

Methods

The core contribution is a "hybrid two-level MCMC" framework. This method uses a two-stage approach:

  1. Base MCMC Chain: A long Markov Chain Monte Carlo (MCMC) chain is run using a fast DL-based surrogate model. This provides a quick, albeit potentially less accurate, approximation of the posterior distribution.
  2. Correction MCMC Chain: A short MCMC chain is then run using a high-fidelity numerical model. This chain is designed to correct the errors introduced by the DL surrogate, effectively "refining" the results from the base chain.

This "telescoping sum" approach, inspired by multilevel MCMC methods, allows for theoretical error bounds to be established, ensuring that the final result is as accurate as if it were computed solely with the high-fidelity model, but at a significantly reduced computational cost.

Datasets

The paper does not introduce new datasets. Instead, it uses data generated from numerical simulations of the forward models for training the DL surrogate models. These simulations are performed using standard numerical methods like the Finite Element Method (FEM) and pseudospectral methods.

Key Insights and Ideas

  • Bridging the Accuracy-Speed Gap: The hybrid approach effectively bridges the gap between fast but potentially inaccurate DL surrogates and slow but accurate numerical models.
  • Theoretical Guarantees: The method provides a theoretical framework for error estimation, allowing for a priori control over the accuracy of the posterior mean estimation. This is a significant improvement over treating DL models as black boxes.
  • Cost-Effective Accuracy: The key insight is that a small number of high-fidelity samples can "correct" a large number of DL surrogate samples, leading to substantial computational savings without sacrificing accuracy.
  • Generalizability: The framework is presented as generally applicable to various surrogate models (not just DL) and potentially adaptable to other MCMC-related methods.

Key Results

The paper demonstrates the effectiveness of the hybrid two-level MCMC method through numerical experiments on three different PDE problems: Poisson, nonlinear reaction-diffusion, and Navier-Stokes equations.

  • Accuracy: The hybrid method achieves estimator errors comparable to those of a plain numerical MCMC method. For instance, in the elliptic problem with a uniform prior, the hybrid method with only 1% numerical samples achieved an error comparable to a full numerical MCMC run.
  • Speedup: Significant computational speedups are observed. For example, in the elliptic problem, the hybrid method achieved speedups of up to 16.59x compared to the numerical MCMC. The exact speedup depends on the problem and the ratio of numerical to DL samples used.
  • Error Correction: The experiments show that even a small percentage of numerical samples in the correction chain can drastically reduce the error compared to using DL surrogates alone.

Limitations and Future Work

  • Posterior Mean Focus: The current framework is primarily designed for computing the posterior mean of Quantities of Interest (QoI). Generating full posterior histograms, as with conventional MCMC, is limited. However, other statistical quantities like variance and quantiles can still be estimated.
  • Theoretical Assumptions: The theoretical analysis relies on assumptions about knowing the exact DL surrogate error and specific constants, which can be challenging to obtain analytically in practice.
  • Optimal Ratio: While the theory suggests an optimal ratio between numerical and DL samples, finding this exact ratio empirically can be non-trivial and application-dependent.
  • Future Work: The authors suggest that the framework can be generalized to other Bayesian inverse problems (not necessarily PDE-governed) and other sampling algorithms like ensemble Kalman filters and sequential Monte Carlo methods.

Summary updated

Abstract

Bayesian inverse problems arise in various scientific and engineering domains, and solving them can be computationally demanding. This is especially the case for problems governed by partial differential equations, where the repeated evaluation of the forward operator is extremely expensive. Recent advances in Deep Learning (DL)-based surrogate models have shown promising potential to accelerate the solution of such problems. However, despite their ability to learn from complex data, DL-based surrogate models generally cannot match the accuracy of high-fidelity numerical models, which limits their practical applicability. We propose a novel hybrid two-level Markov Chain Monte Carlo (MCMC) method that combines the strengths of DL-based surrogate models and high-fidelity numerical solvers to {compute the posterior mean of Quantities of Interest (QoI) in} Bayesian inverse problems governed by partial differential equations. The intuition is to leverage the evaluation speed of a DL-based surrogate model as the base chain, and correct its errors using a limited number of high-fidelity numerical model evaluations in a correction chain; hence its name hybrid two-level MCMC method. Through a detailed theoretical analysis, we show that our approach can achieve the same accuracy as a pure numerical MCMC method while requiring only a small fraction of the computational cost. The theoretical analysis is further supported by several numerical experiments, namely a Poisson, a non-linear reaction-diffusion, and a Navier-Stokes equation. The proposed hybrid framework can be generalized to other approaches such as the ensemble Kalman filter and sequential Monte Carlo methods.
BibTeX
@article{yang2026hybrid,
  author = {Juntao Yang and Jeff Adie and Simon See and Adriano Gualandi and Gianmarco Mengaldo},
  title = {{A hybrid two-level MCMC framework to accelerate posterior mean estimation with deep learning surrogates for bayesian inverse problems}},
  journal = {Journal of Computational Physics},
  year = {2026},
  doi = {10.1016/j.jcp.2025.114502},
}
Figure