GANs and Closures: Micro-Macro Consistency in Multiscale Modeling

Ellis R. Crabtree, Juan M. Bello-Rivas, Andrew L. Ferguson, Ioannis G. Kevrekidis

This paper explores a clever marriage between machine learning and classical physics to solve a long-standing problem in multiscale modeling: how to move seamlessly between "big picture" macroscopic descriptions and "fine-grained" microscopic realities.

TL;DR

The authors propose a framework that couples Conditional Generative Adversarial Networks (cGANs) with physics-based "Umbrella Sampling" to bridge the gap between coarse and fine-scale models. By using cGANs to rapidly "lift" low-dimensional descriptors into realistic high-dimensional states, they achieve significant speedups in sampling complex dynamical systems while maintaining physical rigor.

What Problem Does This Paper Address?

In fields like protein folding or materials science, systems are "multiscale." They are governed by a few "slow" variables (like the overall shape of a molecule) and many "fast" variables (the jittering of individual atoms).

Scientists often use dimensionality reduction to find these slow variables, but the inverse process—lifting—is much harder. Lifting involves taking a single "coarse" state and generating many plausible "fine" realizations of it. Traditional physics-based methods, like Umbrella Sampling (US), are great at this but are computationally expensive and slow to equilibrate. Conversely, Machine Learning (ML) models like GANs are incredibly fast at generating samples but often ignore physical constraints and require massive amounts of training data.

Methods

The paper introduces a "tandem" approach:

  1. Diffusion Maps: If the "slow" variables (Collective Variables or CVs) aren't known, the authors use Diffusion Maps—a non-linear dimensionality reduction technique—to discover them from raw data.
  2. Continuous Conditional GANs (CcGAN): Standard cGANs usually condition on discrete labels (like "cat" vs "dog"). Here, the authors use a CcGAN to condition on continuous physical variables (like a dihedral angle in a molecule).
  3. The Coupling: Instead of letting the GAN have the final word, they use the GAN's output as "warmed-up" initial conditions for parallel, short bursts of physics-based Umbrella Sampling.
  4. Itô’s Lemma: To ensure the physics-based biasing works correctly in the "learned" coordinate space, the authors use this mathematical tool to transform the underlying Stochastic Differential Equations (SDEs).

Datasets

The authors evaluated their framework on three main types of systems:

  • Toy SDE Systems: 2D and 3D stochastic systems designed with non-linear geometries (like "half-moons" and "caps") to test if the GAN could learn complex, curved manifolds.
  • Metastable Systems: A system with high energy barriers to see if the cGAN could help the simulation "jump" between different stable states that are usually hard to reach.
  • Alanine Dipeptide: A standard benchmark in molecular dynamics. This is a 22-dimensional system representing a small protein fragment in a solvent.

Key Insights and Ideas

  • Lifting as Generative Modeling: The core "aha moment" is realizing that the "lifting" step in multiscale modeling is functionally identical to what a GAN does: mapping a low-dimensional label to a high-dimensional distribution.
  • Amortized Computation: Training a GAN is an "up-front" cost. Once trained, it can generate 100,000 samples in less than a second—a task that would take hours or days using traditional simulation.
  • Noise Dimensionality is a Dial: The authors found that the dimensionality of the noise fed into the GAN generator acts as a control. If you give it 1D noise, it might only sample a line; give it 2D noise, and it can capture an entire "surface" of the conditional distribution.
  • Extrapolation Capabilities: Surprisingly, the cGAN showed the ability to produce reasonable (though approximate) fine-scale states for coarse variables outside the range of its training data, acting as a "scout" for exploring new regions of phase space.

Key Results

  • Speedup: The cGAN produced conditional data significantly faster than Umbrella Sampling. In one test, the cGAN sampled 100,000 points in 0.3 seconds, while the equivalent physics-based integration took orders of magnitude longer.
  • Improved Convergence: By using cGAN outputs to initialize 10 parallel US runs, the authors reached an accurate distribution 10 times faster (in wall-clock time) than a single long US simulation.
  • Overcoming Barriers: In systems with high energy barriers (metastability), the coupled approach was far more accurate. While standard US often got "stuck" in one basin, the cGAN provided starting points in multiple basins, allowing the simulation to capture the full physical picture.

Limitations and Future Work

  • Data Dependency: The GAN still needs a decent amount of "unbiased" simulation data to learn the initial distribution.
  • Mode Collapse: While not observed in this study, GANs are notoriously prone to "mode collapse" (where they only generate a few types of samples), which would be a disaster for physical sampling.
  • Scaling Up: While Alanine Dipeptide is a great start, the real test will be scaling this to massive proteins or complex material interfaces where the dimensionality is in the thousands or millions.
  • Diffusion Models: The authors suggest that newer generative techniques, like Diffusion Models, might be even more robust than GANs for this task in the future.

Summary updated

Abstract

Sampling the phase space of molecular systems -- and, more generally, of complex systems effectively modeled by stochastic differential equations -- is a crucial modeling step in many fields, from protein folding to materials discovery. These problems are often multiscale in nature: they can be described in terms of low-dimensional effective free energy surfaces parametrized by a small number of "slow" reaction coordinates; the remaining "fast" degrees of freedom populate an equilibrium measure on the reaction coordinate values. Sampling procedures for such problems are used to estimate effective free energy differences as well as ensemble averages with respect to the conditional equilibrium distributions; these latter averages lead to closures for effective reduced dynamic models. Over the years, enhanced sampling techniques coupled with molecular simulation have been developed. An intriguing analogy arises with the field of Machine Learning (ML), where Generative Adversarial Networks can produce high dimensional samples from low dimensional probability distributions. This sample generation returns plausible high dimensional space realizations of a model state, from information about its low-dimensional representation. In this work, we present an approach that couples physics-based simulations and biasing methods for sampling conditional distributions with ML-based conditional generative adversarial networks for the same task. The "coarse descriptors" on which we condition the fine scale realizations can either be known a priori, or learned through nonlinear dimensionality reduction. We suggest that this may bring out the best features of both approaches: we demonstrate that a framework that couples cGANs with physics-based enhanced sampling techniques can improve multiscale SDE dynamical systems sampling, and even shows promise for systems of increasing complexity.
BibTeX
@article{crabtree2023gans,
  author = {Ellis R. Crabtree and Juan M. Bello-Rivas and Andrew L. Ferguson and Ioannis G. Kevrekidis},
  title = {{GANs and Closures: Micro-Macro Consistency in Multiscale Modeling}},
  journal = {Multiscale Model. Simul.},
  year = {2023},
  doi = {10.1137/22m1517834},
}

Loading related content...

Figure