Uncertainty estimation by committee models for molecular dynamics and thermodynamic averages

Giulio Imbalzano, Yongbin Zhuang, Venkat Kapil, Kevin Rossi, Edgar A. Engel, Federico Grasselli, Michele CeriottiarXiv 2020 · arXiv

Abstract

Machine learning (ML) potentials have revolutionized molecular dynamics (MD) by offering ab initio accuracy at a fraction of the computational cost. However, these models are fundamentally interpolative and can become unreliable when extrapolating into unrepresented atomic configurations. This paper introduces a robust framework using committee models to quantify uncertainty and provides a stable method to propagate these errors into thermodynamic averages, such as free energies and radial distribution functions, using a Cumulant Expansion Approximation (CEA).

Introduction

In molecular dynamics, a simulation may wander into a "blind spot"—a configuration of atoms not represented in the training set. When this occurs, standard ML potentials often provide unphysical forces, causing simulations to fail or yield incorrect results. Furthermore, researchers typically require thermodynamic averages (e.g., melting points or density of states) rather than single-point energies. Estimating how the underlying ML model's uncertainty propagates into these collective properties has historically been computationally prohibitive, often requiring multiple independent, expensive simulations.

Methodology

The authors employ a Committee Model approach, training an ensemble of $M$ models on different subsets of reference data. The variance among the committee's predictions serves as a proxy for uncertainty.

Overview of the committee model framework, from database construction to the calculation of thermodynamic uncertainty.

Key methodological innovations include:

  • Calibration: A scaling factor ($\alpha$) ensures the committee's spread quantitatively matches actual prediction errors on validation sets.
  • Weighted Baseline Scheme: To prevent simulations from failing in unexplored regions, the potential smoothly transitions from the ML model to a robust baseline potential (like DFTB) as uncertainty increases.
  • On-the-fly Reweighting (CEA): To calculate the uncertainty of a thermodynamic average from a single trajectory, the authors utilize a Cumulant Expansion Approximation. This circumvents the "curse of dimensionality" associated with traditional statistical reweighting, allowing for efficient error propagation.

Key Results and Applications

The framework was validated across diverse chemical systems, demonstrating both stability and predictive power.

Thermodynamic Observables in Water

For bulk water, the committee approach predicted a melting point of $290 \pm 5$ K. The error bar represents the inherent uncertainty of the ML model rather than just statistical noise. The CEA method proved highly effective at estimating the uncertainty in the radial distribution function $g(r)$, as shown below.

Radial distribution function g(r) for water (top) and the corresponding uncertainty Δg(r) calculated via different methods (bottom).

Stability and Rare Events

In a Phe-Gly-Phe tripeptide system, the weighted baseline allowed simulations to remain stable at temperatures up to 2440 K, where the molecule fragments. A standard ML potential would have failed immediately in this regime. Additionally, for methanesulfonic acid deprotonation, the framework estimated a free energy of $20^{+5}_{-2}$ kJ/mol, with asymmetric error bars reflecting higher model uncertainty in the rare, dissociated state.

Discussion and Limitations

The "safety net" provided by the weighted baseline is essential for sampling high-energy states or rare events like chemical dissociation. Furthermore, because ML models often decompose energy into atomic contributions, uncertainty can be mapped to specific atoms, facilitating targeted active learning.

However, the framework relies on Gaussian assumptions for committee errors, which may break down in highly non-linear regimes. The authors also recommend a minimum committee size (at least $M=6$) to avoid biased estimates. While CEA is more stable than direct reweighting, its efficiency may still degrade as system size increases, suggesting a future need for localized error propagation techniques.

Summary updated

Abstract

Machine learning models have emerged as a very effective strategy to sidestep time-consuming electronic-structure calculations, enabling accurate simulations of greater size, time scale and complexity. Given the interpolative nature of these models, the reliability of predictions depends on the position in phase space, and it is crucial to obtain an estimate of the error that derives from the finite number of reference structures included during the training of the model. When using a machine-learning potential to sample a finite-temperature ensemble, the uncertainty on individual configurations translates into an error on thermodynamic averages, and provides an indication for the loss of accuracy when the simulation enters a previously unexplored region. Here we discuss how uncertainty quantification can be used, together with a baseline energy model, or a more robust although less accurate interatomic potential, to obtain more resilient simulations and to support active-learning strategies. Furthermore, we introduce an on-the-fly reweighing scheme that makes it possible to estimate the uncertainty in the thermodynamic averages extracted from long trajectories. We present examples covering different types of structural and thermodynamic properties, and systems as diverse as water and liquid gallium.
BibTeX
@misc{imbalzano2020uncertainty,
  author = {Giulio Imbalzano and Yongbin Zhuang and Venkat Kapil and Kevin Rossi and Edgar A. Engel and Federico Grasselli and Michele Ceriotti},
  title = {{Uncertainty estimation by committee models for molecular dynamics and thermodynamic averages}},
  year = {2020},
  eprint = {2011.08828},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2011.08828},
}
Figure