Variational Methods for Posterior Estimation of Non-linear Inverse Problems

Elizabeth Hou, Earl Lawrence2018 · DOI

Introduction

Inverse problems—determining the input $x$ that produced an observed output $y$—are fundamental to scientific inquiry, from geophysics to medical imaging. However, when the "forward" process (the physical experiment or simulation) is non-linear and computationally expensive, traditional Bayesian inference becomes a significant bottleneck. This research by Elizabeth Mary Hou and Earl Christopher Lawrence at Los Alamos National Laboratory introduces a novel variational framework designed to provide fast, analytical posterior estimations for these complex systems.

The Challenge of Non-linear Inverse Problems

Standard approaches to inverse problems often rely on Markov Chain Monte Carlo (MCMC) sampling. While MCMC is asymptotically exact, it is computationally intensive, lacks an analytical form, and can struggle with convergence in high-dimensional or highly non-linear spaces. Furthermore, non-linear forward models frequently result in multi-modal posteriors, where several distinct inputs could plausibly produce the same output. Existing variational methods often fail to capture these multiple "peaks" in the probability distribution, often collapsing to a single mode.

A Novel Variational Framework

The proposed method shifts the focus from sampling to Information Projections. The framework is built upon several key technical pillars:

  1. Principle of Minimum Relative Entropy: The problem is reformulated as finding a distribution that minimizes the relative entropy to the prior while satisfying constraints derived from the observed data. This frames posterior estimation as a constrained optimization problem.
  2. Lagrange-Burmann Inversion: To address non-linearity, the authors utilize the Lagrange-Burmann formula to locally invert the forward function. This allows them to approximate the inverse mapping directly.
  3. Mixture of Experts: The global non-linear inverse mapping is approximated as a mixture of first-order (linear) truncations. This allows the framework to partition the input space into regions where linear approximations are valid, naturally accommodating multi-modality by assigning different "experts" to different regions of the input space.
  4. Optimization via MM Algorithm: A Minorization-Maximization (MM) algorithm is employed to solve for latent variables and Lagrange multipliers. By choosing constraints that are linear in the parameters of the prior, the resulting posterior remains within the same exponential family, making the mathematics tractable and providing a closed-form solution.

Experimental Evaluation

The framework was evaluated using synthetic 1D simulation data generated from a Gaussian Process. The variational posterior was compared against empirical histograms generated via MCMC-style sampling across several test cases (Examples 1–4).

The results demonstrate that the variational posterior provides an excellent fit to the empirical data. Notably, the method successfully identifies correct input regions even when the forward function is highly oscillatory. In cases of multi-modality—where the forward function crosses the observed $y$ value at multiple points—the variational distribution correctly places probability mass at all corresponding $x$ locations. Unlike MCMC, which requires thousands of samples to approximate a distribution, this method yields a continuous analytical formula efficiently.

Conclusion and Future Directions

This research presents a promising alternative to sampling-based posterior estimation for non-linear inverse problems. By leveraging local linearity and information projections, the authors provide a path toward efficient, closed-form solutions that can capture complex, multi-modal distributions. Future work will focus on scaling the framework to higher-dimensional datasets, improving computational stability, and refining the selection process for the number of components in the mixture model.

Summary updated

Abstract

No abstract provided.
BibTeX
@misc{hou2018variational,
  author = {Elizabeth Hou and Earl Lawrence},
  title = {{Variational Methods for Posterior Estimation of Non-linear Inverse Problems}},
  year = {2018},
  doi = {10.2172/1475317},
}
Figure