Neural networks-based variationally enhanced sampling

Luigi Bonati, Yueyu Zhang, Michele ParrinelloProceedings of the National Academy of Sciences 2019 · DOI

TL;DR

This paper introduces DEEP-VES, a novel method that leverages neural networks within the framework of Variationally Enhanced Sampling (VES) to accelerate the exploration of complex molecular free energy landscapes. By representing the bias potential as a neural network, DEEP-VES can efficiently capture intricate free energy surfaces and handle multiple collective variables, overcoming limitations of traditional VES approaches.

What Problem Does This Paper Address?

Simulating molecular systems often involves navigating complex "free energy surfaces" that describe the energy landscape as a function of various molecular configurations. A major challenge arises when these surfaces have "kinetic bottlenecks" – energy barriers that prevent the system from easily transitioning between important states. Traditional simulation methods can get trapped in local minima, making it incredibly difficult and time-consuming to explore the full range of relevant configurations and accurately calculate free energy differences.

Existing enhanced sampling methods, like Variationally Enhanced Sampling (VES), attempt to overcome this by introducing a "bias potential" that guides the simulation towards exploring these challenging regions. However, traditional VES methods often rely on expanding this bias potential using predefined basis functions, which can become computationally expensive and struggle to accurately represent complex, rapidly changing free energy landscapes, especially when multiple "collective variables" (simplified descriptors of the system's state) are needed.

Methods

The core innovation of this paper is the integration of neural networks (NNs) into the VES framework, creating what they call DEEP-VES. Instead of using a linear combination of basis functions to represent the bias potential $V(s)$, they use a deep neural network.

Here's a breakdown of the key methodological aspects:

  • Neural Network for Bias: The collective variables ($s$) are fed as input to a neural network. The network's output is the bias potential ($V(s)$). This allows for a highly flexible and expressive representation of the bias, capable of learning complex, non-linear relationships.
  • Variational Principle: The parameters of the neural network (weights and biases) are optimized using a variational principle, aiming to minimize a functional ($\Omega[V]$) that is related to the Kullback-Leibler divergence between the biased and a target probability distribution. This ensures that the biased system samples configurations according to a desired distribution.
  • Stochastic Optimization: A novel, efficient stochastic optimization scheme is developed to train the neural network parameters. This involves a "stochastic steepest descent" approach, where the gradients are computed using back-propagation. The optimization is performed "on-the-fly" during the simulation, meaning the network is updated as new data is generated, without needing to stop and refit.
  • Target Distribution: The paper utilizes a "well-tempered" target distribution, inspired by well-tempered metadynamics. This approach helps to accelerate convergence and is particularly useful when the true free energy surface is not known beforehand.
  • Handling Multiple Collective Variables: The expressivity of neural networks allows DEEP-VES to naturally handle bias potentials that depend on multiple collective variables, a significant advantage over traditional methods that scale poorly with dimensionality.

Datasets

The paper does not introduce new benchmark datasets. Instead, it evaluates its DEEP-VES method on three established systems:

  • Wolfe-Quapp potential: A 2D toy model used to demonstrate the method's ability to handle sub-optimal collective variables and rapidly varying potentials.
  • Alanine Dipeptide and Tetrapeptide: Small peptide molecules, commonly used to test enhanced sampling techniques, allowing evaluation on systems with 2 and 6 collective variables, respectively.
  • Silicon Crystallization: A more complex, industrially relevant problem involving a high energy barrier, used to showcase the method's ability to handle challenging phase transitions.

Key Insights and Ideas

  • Neural Networks as Universal Approximators for Bias: The paper demonstrates that neural networks are powerful tools for representing complex bias potentials in enhanced sampling. Their inherent flexibility allows them to learn intricate free energy landscapes that might be difficult to capture with traditional basis sets.
  • On-the-Fly Optimization for Efficiency: Training the neural network parameters directly during the simulation, rather than as a separate post-processing step, significantly streamlines the workflow and allows for adaptive bias refinement.
  • Handling High-Dimensionality: The ability of NNs to scale with the number of collective variables is a crucial advantage, opening doors to more accurate sampling of systems where simple 1D or 2D collective variables are insufficient.
  • Bridging the Gap between ML and Physics: This work effectively bridges the gap between machine learning techniques (specifically neural networks) and fundamental physics simulations, showing how ML can be directly applied to solve challenging problems in computational physics.

Key Results

  • Accurate Free Energy Surfaces: DEEP-VES successfully recovers accurate free energy profiles for all tested systems, comparable to or exceeding the accuracy of established methods like reweighting and metadynamics.
  • Efficient Sampling: The method demonstrates efficient exploration of complex free energy landscapes, including those with high energy barriers and sub-optimal collective variables. For instance, in the Alanine Dipeptide case, the KL divergence threshold was reached after 3 ns, and the bias was no longer updated after 12 ns.
  • Handling Multiple CVs: The Alanine Tetrapeptide example, using six dihedral angles as collective variables, showcases DEEP-VES's capability in dealing with higher-dimensional free energy landscapes.
  • Robustness: The method is shown to be robust with respect to parameter choices, particularly when good collective variables are identified.

Limitations and Future Work

  • Sub-optimal CVs: While DEEP-VES can handle sub-optimal collective variables, the paper notes that the accuracy and convergence speed can be affected. In such cases, the reweighting procedure is still crucial for refining the free energy surface.
  • Parameter Tuning: Although the method is robust, some parameters (KL divergence timescale, threshold, learning rate decay time) still require setting. The paper suggests that for higher-dimensional FES, increasing the KL divergence threshold might be beneficial.
  • Future Directions: The authors propose several exciting avenues for future research:
  • Developing more efficient schemes to exploit the variational principle within NNs.
  • Learning not only the bias potential but also the collective variables themselves "on-the-fly."
    • Extending the method to even higher-dimensional landscapes where the power of NNs can be fully realized.
  • Exploring the application of this approach to other areas of machine learning and neural networks for improving enhanced sampling.

Summary updated

Abstract

Sampling complex free-energy surfaces is one of the main challenges of modern atomistic simulation methods. The presence of kinetic bottlenecks in such surfaces often renders a direct approach useless. A popular strategy is to identify a small number of key collective variables and to introduce a bias potential that is able to favor their fluctuations in order to accelerate sampling. Here, we propose to use machine-learning techniques in conjunction with the recent variationally enhanced sampling method [O. Valsson, M. Parrinello, <i>Phys. Rev. Lett.</i> 113, 090601 (2014)] in order to determine such potential. This is achieved by expressing the bias as a neural network. The parameters are determined in a variational learning scheme aimed at minimizing an appropriate functional. This required the development of a more efficient minimization technique. The expressivity of neural networks allows representing rapidly varying free-energy surfaces, removes boundary effects artifacts, and allows several collective variables to be handled.
BibTeX
@inproceedings{bonati2019neural,
  author = {Luigi Bonati and Yueyu Zhang and Michele Parrinello},
  title = {{Neural networks-based variationally enhanced sampling}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2019},
  doi = {10.1073/pnas.1907975116},
}
Figure