Adaptive Monte Carlo augmented with normalizing flows

Marylou Gabrié, Grant M. Rotskoff, Eric Vanden‐EijndenProceedings of the National Academy of Sciences 2022 · DOI · arXiv

Abstract: Monte Carlo methods are essential for sampling from probability distributions in physical sciences and statistics. However, computational costs often become prohibitive due to slow mixing between modes. We present a framework that formalizes Monte Carlo augmented with normalizing flows, demonstrating that generative models trained on-the-fly with limited prior data can substantially accelerate sampling in complex systems.

Introduction

Sampling from high-dimensional, multi-modal probability distributions is a cornerstone of computational science, yet it remains plagued by the "metastability" problem. In fields ranging from molecular dynamics to Bayesian statistics, standard Markov Chain Monte Carlo (MCMC) algorithms like Langevin dynamics often become trapped in local minima (modes), failing to explore the full state space within reasonable timeframes. This paper introduces a hybrid approach that combines the precision of local MCMC with the global reach of generative machine learning.

Methodology: Concurrent Sampling and Training

The authors propose an adaptive MCMC algorithm that uses Normalizing Flows (NFs)—invertible neural networks—to propose non-local jumps between isolated basins of a distribution.

The Hybrid Kernel

The algorithm alternates between standard local MCMC steps, which explore the fine details of a mode, and "resampling" steps. In the resampling phase, the NF proposes a completely new configuration. Because the NF learns the global structure of the target distribution, these proposals can jump across massive energy barriers that would otherwise stop a local walker.

On-the-Fly Adaptation

A critical innovation is the "concurrent sampling and training" framework. Rather than requiring a massive pre-existing dataset, the NF is updated continuously using a forward Kullback-Leibler (KL) divergence estimated from the samples currently being generated by the MCMC chains. This solves the "chicken-and-egg" problem of needing samples to train a model that is intended to generate those very samples.

Physics-Informed Priors

To combat the curse of dimensionality, the authors use "informed" base measures. Instead of mapping from pure white noise, the NF transforms a distribution that already captures basic physical properties (like short-range correlations). By learning only the deviations from a simple physical model, the neural network's task is significantly simplified.

Training progress and sampling performance. (a) Negative log-likelihood decreases as the flow learns. (b) Acceptance ratio for global jumps increases significantly. (c) Integrated autocorrelation time drops as mixing improves. (d) Sample paths at the end of training show exploration of multiple modes.

Key Results

The method was evaluated on challenging benchmarks, including the stochastic Allen-Cahn model and interacting particle systems undergoing phase transitions.

Rapid Mixing and Efficiency

As shown in the training metrics, the adaptive sampler achieved an acceptance rate of approximately 60% for global jumps in the Allen-Cahn model. The integrated autocorrelation time—a measure of how many steps are needed to generate an independent sample—collapsed as the NF matured. This allowed a single walker to transition between modes in just 10 steps, a feat that would take millions of iterations for traditional MCMC.

Accurate Free Energy Estimation

Beyond generating samples, the trained flows can be used via importance sampling to calculate thermodynamic quantities like free energy differences.

Free energy difference estimation using Normalizing Flow Importance Sampling (NF IS) compared to Laplace approximations across different external fields h and inverse temperatures beta.

As illustrated in the comparison with analytical approximations, the NF-based approach provides highly accurate estimates of free energy differences ($F_+ - F_-$) across various external fields ($h$) and temperatures ($eta$), outperforming standard second-order Laplace approximations.

Conclusion and Future Work

This hybrid approach effectively bridges the gap between traditional sampling and generative modeling. While the algorithm currently requires initial representative configurations for each mode to begin training, it offers a scalable path toward efficient sampling in complex physical systems. Future research will focus on scaling to even higher dimensions using patch-based flows and handling systems with hardcore repulsions.

Summary updated

Abstract

SignificanceMonte Carlo methods, tools for sampling data from probability distributions, are widely used in the physical sciences, applied mathematics, and Bayesian statistics. Nevertheless, there are many situations in which it is computationally prohibitive to use Monte Carlo due to slow "mixing" between modes of a distribution unless hand-tuned algorithms are used to accelerate the scheme. Machine learning techniques based on generative models offer a compelling alternative to the challenge of designing efficient schemes for a specific system. Here, we formalize Monte Carlo augmented with normalizing flows and show that, with limited prior data and a physically inspired algorithm, we can substantially accelerate sampling with generative models.
BibTeX
@inproceedings{gabrie2022adaptive,
  author = {Marylou Gabrié and Grant M. Rotskoff and Eric Vanden‐Eijnden},
  title = {{Adaptive Monte Carlo augmented with normalizing flows}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2022},
  doi = {10.1073/pnas.2109420119},
}
Figure