Δ 2 machine learning for reaction property prediction

Qiyuan Zhao, Dylan M. Anstine, Olexandr Isayev, Brett M. SavoieChemical 2023 · DOI

TL;DR

This paper introduces a novel "Δ² machine learning" (Delta-squared ML) approach that accurately predicts high-level chemical reaction properties (like activation energies) using low-level computational geometries. This method significantly accelerates reaction discovery and characterization by achieving near-DFT accuracy at a fraction of the computational cost, comparable to faster semi-empirical methods.

What Problem Does This Paper Address?

Accurately predicting chemical reaction properties, such as activation energies (the energy barrier a reaction must overcome), is crucial for understanding reaction mechanisms, designing catalysts, and predicting reaction outcomes. However, the most accurate methods for calculating these properties, like high-level density functional theory (DFT), are computationally very expensive. This cost limits their application to large-scale automated reaction discovery or the exploration of complex reaction networks.

Traditional "Δ-learning" (Delta-learning) methods try to address this by training ML models to predict the correction needed to a low-level energy calculation to reach a high-level energy at a given geometry. However, this approach is insufficient for reaction properties because it requires both the geometry and the energy at the high level of theory, and the high-level geometry itself is often the bottleneck. The problem, therefore, is how to predict accurate high-level reaction properties without performing expensive high-level geometry optimizations for every single reaction.

Methods

The core innovation is the Δ² machine learning (Delta-squared ML) model. Unlike standard Δ-learning, which predicts energy differences at a fixed geometry, Δ² ML predicts the high-level energy at the high-level critical point (e.g., transition state) using only the low-level optimized geometry and energy.

Here's how it works:

  • Low-Level Input: The model takes geometries and energies calculated using a fast, low-level method (GFN2-xTB in this study). These are the "critical points" on the potential energy surface, which can be reactants, products, or transition states.
  • Implicit Geometry Learning: The "Δ²" in the name signifies that the model implicitly learns the difference in geometry (Δ geometry) between the low-level and high-level calculations, in addition to learning the difference in energy (Δ energy). This is a key departure from previous methods.
  • Atom-Wise Featurization: The model uses an atom-wise featurization approach, similar to modern ML interatomic potentials (MLIPs), allowing it to capture local chemical environments.
  • Neural Network Architecture: The specific ML model used is a variant of the AIMNet2-NSE architecture, which employs a message-passing neural network. This architecture is adept at learning complex relationships between atomic environments.
  • Transfer Learning: The authors also demonstrate that fine-tuning the Δ² model with a small amount of data calculated at an even higher level of theory (Gaussian-4) can further boost accuracy.

Datasets

  • RGD1 Dataset: This is the primary dataset used for training and testing. It contains approximately 210,000 distinct reactions, generated through a graph-based exploration without relying on predefined reaction types. The RGD1 dataset is notable for its diversity and inclusion of reactions with up to ten heavy atoms.
  • External Test Sets:
    • EXT1: Three unimolecular decomposition networks (g-ketohydroperoxide, propylene carbonate, and methyl butanoate) with 324 reactions.
  • EXT2: 118 reactions from the first step of L-glucose pyrolysis. This set is significant because it includes larger molecules (more than ten heavy atoms) and more complex chemistry than typically found in RGD1, testing the model's transferability.

Key Insights and Ideas

  • Implicit Geometry Learning is Key: The most significant insight is that the Δ² model's ability to implicitly learn the geometric differences between low-level and high-level calculations is crucial for its success. This allows it to predict accurate high-level energies even when the low-level geometries are significantly different from the true high-level geometries.
  • Beyond Standard Δ-Learning: The paper clearly demonstrates the limitations of traditional Δ-learning for reaction properties and introduces a more powerful framework that addresses this gap.
  • High Accuracy at Low Cost: The Δ² model achieves accuracy comparable to high-level DFT calculations but at a computational cost similar to the low-level GFN2-xTB method. This is a major breakthrough for accelerating chemical research.
  • Transferability is Surprisingly Good: The model shows remarkable transferability to unseen reactions and even to slightly larger molecules, indicating a robust underlying learning capability.
  • Handling Complex Reactions: The Δ² model can effectively handle reactions with chemically equivalent reactants/products (ΔH=0) and multiple transition state conformations, which are challenging for simpler ML models.

Key Results

  • Main Test Set Performance: The Δ² model achieved a Mean Absolute Error (MAE) of 1.3 kcal mol⁻¹ on the withheld RGD1 testing set, a significant improvement over direct GFN2-xTB predictions (MAE of 11.9 kcal mol⁻¹).
  • Error Distribution: 99.3% of predicted activation energies were within a 10 kcal mol⁻¹ deviation range of the DFT target.
  • Transfer Learning Boost: Fine-tuning the Δ² model with just 2000 Gaussian-4 (G4) calculations resulted in a 35% accuracy improvement compared to direct DFT calculations on those G4 test data.
  • External Test Set Performance: On the EXT1 set, the MAE was 1.52 kcal mol⁻¹, and on EXT2, it was 2.33 kcal mol⁻¹. While slightly higher than the main test set, these results are still impressive and demonstrate near "chemical accuracy" (<1 kcal mol⁻¹).
  • Computational Cost: The runtime of the Δ² model is negligible compared to the GFN2-xTB calculation, making it highly efficient.

Limitations and Future Work

  • Size Limitation: The current accurate performance is primarily for reactions with up to ten C, N, and O atoms. Extending this to larger and more diverse reaction spaces is a key future direction.
  • Neutral Closed-Shell Species: The model is currently trained on neutral, closed-shell species. Extending it to ionic and open-shell molecules is a natural next step, though it will require new data curation and benchmarking.
  • Data Curation: The authors suggest that more extensive and complex reaction databases, or ad-hoc transfer learning approaches, will be needed to further improve accuracy and generality.
  • Model Refinement: While the Δ² model is highly effective, further improvements in model architecture and training strategies could lead to even better performance.

Summary updated

Abstract

The emergence of Δ-learning models, whereby machine learning (ML) is used to predict a correction to a low-level energy calculation, provides a versatile route to accelerate high-level energy evaluations at a given geometry. However, Δ-learning models are inapplicable to reaction properties like heats of reaction and activation energies that require both a high-level geometry and energy evaluation. Here, a Δ<sup>2</sup>-learning model is introduced that can predict high-level activation energies based on low-level critical-point geometries. The Δ<sup>2</sup> model uses an atom-wise featurization typical of contemporary ML interatomic potentials (MLIPs) and is trained on a dataset of ∼167 000 reactions, using the GFN2-xTB energy and critical-point geometry as a low-level input and the B3LYP-D3/TZVP energy calculated at the B3LYP-D3/TZVP critical point as a high-level target. The excellent performance of the Δ<sup>2</sup> model on unseen reactions demonstrates the surprising ease with which the model implicitly learns the geometric deviations between the low-level and high-level geometries that condition the activation energy prediction. The transferability of the Δ<sup>2</sup> model is validated on several external testing sets where it shows near chemical accuracy, illustrating the benefits of combining ML models with readily available physical-based information from semi-empirical quantum chemistry calculations. Fine-tuning of the Δ<sup>2</sup> model on a small number of Gaussian-4 calculations produced a 35% accuracy improvement over DFT activation energy predictions while retaining xTB-level cost. The Δ<sup>2</sup> model approach proves to be an efficient strategy for accelerating chemical reaction characterization with minimal sacrifice in prediction accuracy.
BibTeX
@inproceedings{zhao2023sup,
  author = {Qiyuan Zhao and Dylan M. Anstine and Olexandr Isayev and Brett M. Savoie},
  title = {{Δ <sup>2</sup> machine learning for reaction property prediction}},
  booktitle = {Chemical},
  year = {2023},
  doi = {10.1039/d3sc02408c},
}
Figure