Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics

Guoqing Zhou, Nicholas Lubbers, Kipton Barros, Sergei Tretiak, Benjamin NebgenProceedings of the National Academy of Sciences 2022 · DOI

TL;DR

This paper introduces a novel machine learning approach that dynamically adjusts the parameters of semiempirical quantum mechanics (SEQM) methods based on the local atomic environment. This "HIPNN+SEQM" model significantly improves the accuracy, extensibility, and transferability of SEQM calculations, achieving near quantum-level accuracy with much greater efficiency than traditional methods.

What Problem Does This Paper Address?

Computational chemistry often faces a trade-off between accuracy and computational cost. High-accuracy methods like Density Functional Theory (DFT) are computationally expensive, limiting their application to small systems or requiring significant computational resources. Machine learning (ML) models have emerged as a way to achieve high accuracy at lower costs by learning to predict molecular properties directly from data. However, many existing ML models suffer from poor "extensibility" and "transferability"—meaning their accuracy drops significantly when applied to molecules or chemical environments not seen in their training data. This limits their general applicability. This paper aims to bridge this gap by integrating the physics-based framework of SEQM with the learning power of deep neural networks to create models that are both accurate and broadly applicable.

Methods

The core innovation is the HIPNN+SEQM model. It combines a deep neural network architecture called HIPNN (Hierarchical Interacting-Particle Neural Network) with a Semiempirical Quantum Mechanics (SEQM) framework, specifically using the Parametric Method 3 (PM3) as the base.

Here's how it works:

  • HIPNN as a Dynamic Parameterizer: Instead of learning to predict molecular properties directly, HIPNN learns to predict the parameters of the SEQM Hamiltonian. These parameters, which are typically static in traditional SEQM, are now dynamically adjusted based on the local chemical environment of each atom. This is achieved by feeding molecular configurations into HIPNN, which then outputs corrections to SEQM parameters.
  • SEQM Core: The SEQM module (implemented in PYSEQM) takes these dynamically generated parameters and performs a standard self-consistent field (SCF) calculation to obtain molecular energies and forces. This retains the physics-based structure of SEQM, including its treatment of electron interactions and nuclear repulsion.
  • Physics-Informed ML: By incorporating the SEQM framework, the ML model is guided by established quantum mechanical principles. This is a departure from purely "heuristic" ML models that learn correlations without explicit physical grounding.
  • Backpropagation through SCF: The authors enabled backpropagation through the SCF procedure, allowing the entire model (HIPNN and SEQM) to be trained end-to-end using molecular energies and forces as targets, referencing DFT calculations.
  • PM3* Model: For comparison, the authors also created a "PM3*" model by reoptimizing the static PM3 parameters using ML tools to best fit the DFT reference data.

Datasets

  • Training Data: The model was trained on approximately 60,000 small organic molecular conformers. A subset of the ANI-1x dataset (61,842 samples) was used for training and validation.
  • Testing/Evaluation Data:
    • COMP6 Benchmark Dataset: This is a comprehensive dataset designed to test model transferability. It includes various subsets:
  • GDB07to09 and GDB10to13 (subsets of the Chemical Universe Generated Database) * Drug Bank subset * Tripeptide subset * ANI-MD subset (from molecular dynamics simulations) * S66x8 subset (focusing on noncovalent interactions)
    • Drug Bank Subset (for optimization): Used to test the prediction of optimized structures and vibrational spectra.

Key Insights and Ideas

  • Dynamic SEQM Parameters are Key: The most significant insight is that making SEQM parameters environment-dependent, rather than static, dramatically improves accuracy and transferability. This allows the model to adapt to diverse chemical environments.
  • Interpretability through Physics: By retaining the SEQM framework, the learned parameters (e.g., "orbital energy," "orbital radial exponent") have physical meaning. The study shows these parameters correlate with atomic hybridization and bonding order, providing a degree of interpretability often lacking in black-box ML models.
  • Extensibility Beyond Training Data: The HIPNN+SEQM model shows remarkable performance on systems significantly larger than those in the training set, a major limitation of many pure ML approaches. This suggests that physics-informed ML can generalize better.
  • Bridging the Accuracy-Cost Gap: The model achieves accuracy comparable to DFT for many properties but at a fraction of the computational cost, making it suitable for larger-scale simulations like molecular dynamics.
  • Improved Handling of Nonequilibrium Configurations: The model demonstrates robust performance even at high temperatures, suggesting its utility in simulating dynamic processes where molecules deviate significantly from equilibrium geometries.

Key Results

  • Accuracy Improvement: HIPNN+SEQM reduces energy errors by over 70% compared to the reoptimized PM3* model on held-out test sets. It achieves a mean absolute error (MAE) for atomization energy per atom of 0.0049 eV (0.15 kcal/mol per atom) on the held-out test set.
  • Transferability: On the COMP6 benchmark, HIPNN+SEQM consistently outperforms pure ML (HIPNN) and static SEQM (PM3, PM3*) models, especially for larger molecules. HIPNN alone shows a significant accuracy drop on larger datasets, while HIPNN+SEQM maintains its performance.
  • Force Prediction: HIPNN+SEQM shows significantly lower root-mean-square errors (RMSE) and MAE for atomic forces compared to other models, particularly on challenging subsets like Tripeptides.
  • Local Properties: The model accurately predicts bond lengths, angles, and vibrational frequencies, correcting known biases in traditional PM3.
  • HOMO-LUMO Gap Prediction: While not explicitly trained on HOMO-LUMO gaps, HIPNN+SEQM performs nearly as well as PM3, demonstrating the model's ability to retain and apply knowledge across different property types.
  • Data Efficiency: The model achieves high performance with a relatively small training dataset (around 60,000 conformers), significantly less than what is typically required for pure ML models.

Limitations and Future Work

  • SEQM Drawbacks: The paper acknowledges that HIPNN+SEQM still inherits some limitations of SEQM, such as poorer performance for certain torsion rotations and hydrogen bonding cases, despite the inclusion of dispersion corrections.
  • Long-Range Interactions: While improved, the model's performance on long-range noncovalent interactions might be limited by the training data's emphasis on short-range interactions.
  • Future Directions:
    • Augmenting training data with active learning or using more sophisticated semiempirical methods (e.g., OMx, GFN2-xTB) could further improve performance.
  • Applying this methodology to study excited-state dynamics and chemical reactions, where accurate and efficient electronic structure calculations are crucial.
  • Explicitly training the model on excited-state quantities to improve predictions in that domain.
  • Further exploring the interpretability of the learned parameters and their correlation with chemical phenomena.

Summary updated

Abstract

Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that are simultaneously accurate, transferable, and interpretable.
BibTeX
@inproceedings{zhou2022deep,
  author = {Guoqing Zhou and Nicholas Lubbers and Kipton Barros and Sergei Tretiak and Benjamin Nebgen},
  title = {{Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2022},
  doi = {10.1073/pnas.2120333119},
}
Figure