Active Learning for Saddle Point Calculation

Xiang Zhou 周翔, Shuting Gu, Hongqiao Wang, Xiang Zhou

TL;DR

This paper introduces an "active learning" approach to efficiently find saddle points (transition states) in complex energy landscapes, which are crucial for understanding chemical reactions. By intelligently selecting where to gather expensive data, their method significantly reduces the computational cost compared to traditional techniques.

What Problem Does This Paper Address?

Calculating saddle points (SPs) on potential energy surfaces (PES) is a fundamental but computationally challenging task in theoretical chemistry. SPs often represent transition states (TSs), which are key to understanding reaction mechanisms and rates. Traditional methods for finding SPs require evaluating the energy function and its derivatives (gradients and Hessians) at a very large number of points. These evaluations can be extremely expensive, especially for large systems or when dealing with free energy surfaces that require lengthy molecular dynamics simulations. This computational bottleneck limits the applicability of these methods to complex systems. While machine learning (ML) surrogate models have been proposed to speed this up, efficiently selecting the data points to train these models remains a challenge.

Methods

The authors propose an adaptive Gaussian Process Regression-Gentlest Ascent Dynamics (aGPR-GAD) method. This approach combines:

  • Gentlest Ascent Dynamics (GAD): A continuous dynamics method for finding saddle points. It uses the gradient and Hessian of the energy function.
  • Gaussian Process Regression (GPR): A powerful non-parametric Bayesian method used to build a surrogate model that approximates the energy function and its derivatives. GPR provides uncertainty estimates, which are crucial for active learning.
  • Active Learning (Optimal Experimental Design): The core innovation. Instead of randomly sampling or using fixed strategies, the method adaptively selects new data points to query. This selection is framed as an optimal experimental design problem, aiming to maximize the information gained about the GAD trajectory. Specifically, they use a criterion based on mutual information to choose "design locations" that maximally reduce the uncertainty of the GPR surrogate model around the evolving GAD trajectory.

The process is sequential: GAD explores the energy landscape, and when the GPR surrogate's reliability drops (indicated by high uncertainty), active learning selects new points to query the true energy function, retrains the GPR, and the GAD continues.

Datasets

The paper uses three examples for evaluation:

  1. A mathematical gradient system: A known energy function with three local minima and two saddle points. This serves as a controlled environment to validate the method against exact solutions.
  2. A two-dimensional non-gradient system: This example tests the method's robustness when measurements are noisy force fields, not just energy values.
  3. Alanine Dipeptide model: A 22-dimensional molecular dynamics problem, representing a computationally intensive, real-world benchmark for studying isomerization processes. The free energy landscape is obtained from molecular dynamics simulations.

No new benchmark datasets were introduced; existing models and simulation data were used.

Key Insights and Ideas

  • Active learning for trajectory optimization: The key insight is that for SP search, the most informative data points are those that lie along the future trajectory of the search algorithm (GAD), not necessarily those that best represent the entire energy landscape. This targeted approach drastically improves efficiency.
  • Uncertainty-driven exploration: GPR's inherent uncertainty estimates are leveraged to identify regions where the surrogate model is unreliable, guiding the active learning process to gather crucial data.
  • Bridging GAD and GPR: The method elegantly integrates a dynamics-based search (GAD) with a data-driven surrogate model (GPR), allowing the surrogate to guide the search and the search to guide data acquisition.
  • Computational cost reduction: The primary benefit is a significant reduction in the number of expensive function evaluations required to find SPs, making complex simulations more feasible.

Key Results

  • Significant speedup: Across all examples, the aGPR-GAD method achieved substantial reductions in computational cost (measured by the number of true energy/force evaluations) compared to classical GAD.
  • In the gradient system example, costs were reduced from hundreds or thousands of evaluations to around 50-110.
  • In the non-gradient system, costs were reduced from thousands to tens of evaluations.
  • For the alanine dipeptide, the reduction was even more dramatic, with aGPR-GAD requiring only 1-2% of the calls to the free energy function compared to classical GAD.
  • Robustness: The method demonstrated robustness even with noisy measurements (non-gradient system example).
  • Accuracy: While the surrogate model might deviate from the true energy landscape in regions far from the trajectory, it accurately approximates the energy around the GAD trajectory, leading to successful SP identification.

Limitations and Future Work

  • Hyper-parameter tuning: While the paper mentions that most hyper-parameters are trained automatically, some (like the threshold for surrogate unreliability) might still require careful setting.
  • Dimensionality: Gaussian processes can become computationally expensive in very high dimensions. The paper notes that Bayesian neural networks or ensembles might be better suited for higher-dimensional problems.
  • Future Work: The authors suggest exploring other surrogate models (like Bayesian neural networks) and potentially more advanced active learning strategies for even higher dimensional or more complex systems. The current work focuses on sequential updates, and exploring batch updates or more sophisticated criteria could be future directions.

Summary updated

Abstract

The saddle point (SP) calculation is a grand challenge for computationally intensive energy function in computational chemistry area, where the saddle point may represent the transition state (TS). The traditional methods need to evaluate the gradients of the energy function at a very large number of locations. To reduce the number of expensive computations of the true gradients, we propose an active learning framework consisting of a statistical surrogate model, Gaussian process regression (GPR) for the energy function, and a single-walker dynamics method, gentle accent dynamics (GAD), for the saddle-type transition states. SP is detected by the GAD applied to the GPR surrogate for the gradient vector and the Hessian matrix. Our key ingredient for efficiency improvements is an active learning method which sequentially designs the most informative locations and takes evaluations of the original model at these locations to train GPR. We formulate this active learning task as the optimal experimental design problem and propose a very efficient sample-based sub-optimal criterion to construct the optimal locations. We show that the new method significantly decreases the required number of energy or force evaluations of the original model.
BibTeX
@article{2022active,
  author = {Xiang ZHOU 周翔 and Shuting Gu and Hongqiao Wang and Xiang Zhou},
  title = {{Active Learning for Saddle Point Calculation}},
  journal = {J. Sci. Comput},
  year = {2022},
  doi = {10.1007/s10915-022-02040-1},
}
Figure