Informing geometric deep learning with electronic interactions to accelerate quantum chemistry

Zhuoran Qiao, Anders S. Christensen, Matthew Welborn, Frederick R. Manby, Anima Anandkumar, Thomas F. Miller

TL;DR

This paper introduces OrbNet-Equi, a novel deep learning framework that integrates electronic interactions into geometric deep learning models for quantum chemistry. By leveraging efficient tight-binding simulations and an equivariant neural network, OrbNet-Equi achieves high accuracy in predicting molecular properties with significantly reduced computational cost and data requirements, paving the way for accelerated discovery in chemistry and materials science.

What Problem Does This Paper Address?

Discovering new molecules and materials is crucial for advancements in areas like drug discovery and energy storage. However, the vastness of chemical space makes experimental exploration and even traditional high-fidelity quantum chemistry simulations (like DFT) computationally prohibitive. Existing machine learning (ML) approaches often struggle with transferability to new chemical spaces due to limited training data and a lack of physical grounding, particularly in capturing electronic degrees of freedom. This paper aims to bridge this gap by developing an ML model that is both data-efficient and physically informed, capable of accurately predicting diverse quantum chemical properties across a wide range of chemical systems.

Methods

The core of this work is the OrbNet-Equi method, built upon a novel neural network architecture called UNiTE. Key methodological aspects include:

  • QM-informed Featurization: Instead of relying solely on geometric descriptors, OrbNet-Equi uses features derived from a coarse-grained quantum mechanics (QM) simulation, specifically tight-binding (GFN-xTB) calculations. These features are based on atomic orbital representations, capturing electronic interactions.
  • Equivariant Neural Network (UNiTE): The UNiTE network is designed to be equivariant to isometric transformations (rotations, translations, and parity inversions) of the molecular system. This ensures that predictions are independent of the viewpoint and respect fundamental physical symmetries. This is achieved through a novel architecture that processes atomic orbital features and updates representations iteratively using block convolution, message passing, and point-wise interaction modules.
  • Incorporating Electronic Interactions: The model explicitly learns from electronic interactions by featurizing based on atomic orbitals and their interactions. For specific properties like frontier molecular orbital energies, they introduce energy-weighted density matrices to better capture electron excitation phenomena.
  • Pooling Layers: Task-specific pooling layers are designed to output predictions for various properties (energies, dipole moments, electron densities, etc.) while respecting relevant physical constraints like charge neutrality.

Datasets

The paper utilizes several established datasets for training and evaluation:

  • QM9: A widely used dataset of 134k small organic molecules with DFT-computed properties, used for benchmarking various targets including atomization energies, dipole moments, and frontier orbital energies.
  • BFDb-SSI: A dataset of amino-acid side-chain dimers, used for evaluating electron density prediction, particularly in cases of charge transfer.
  • Hutchison dataset: A dataset of conformers for ~700 molecules, used to assess transferability for ranking conformer energies.
  • TorsionNet500 dataset: Used to evaluate the prediction of intra-molecular torsion energy profiles.
  • S66x10 dataset: Used to assess the model's ability to describe inter-molecular non-covalent interactions.
  • ROT34 and MCONF datasets: Used for geometry optimization experiments.
  • G21IP dataset: Used to evaluate adiabatic ionization potentials, testing zero-shot generalization.
  • GMTKN55 dataset: A comprehensive benchmark collection for thermochemistry, kinetics, and non-covalent interactions, used to test performance across diverse main-group chemical processes.

The authors also mention using "readily available small-molecule libraries" to curate a training set of 236k molecules for the OrbNet-Equi/SDC21 model, highlighting its ability to learn from diverse, accessible data.

Key Insights and Ideas

  • Physics-Informed ML for Data Efficiency: By incorporating electronic structure information (atomic orbitals and their interactions) and physical symmetries (equivariance) into the deep learning architecture, OrbNet-Equi achieves remarkable data efficiency. It can learn complex properties with significantly less training data compared to traditional ML methods.
  • Equivariance is Key for Transferability: The explicit enforcement of equivariance ensures that the model's predictions are invariant to the system's orientation and position, leading to improved transferability to unseen chemical environments and molecular structures.
  • Atomic Orbitals as a Powerful Representation: Moving beyond purely geometric descriptors, using atomic orbital-based features provides a richer representation that captures the underlying electronic interactions crucial for accurate quantum chemistry predictions.
  • Bridging the Gap Between QM and ML: OrbNet-Equi demonstrates a successful hybrid approach, combining the speed and scalability of ML with the physical rigor and accuracy of QM, offering a promising path towards accelerating scientific discovery.
  • Zero-Shot Generalization: The model shows potential for zero-shot generalization to unseen electronic states, as evidenced by its performance on adiabatic ionization potentials without explicit training on radical species.

Key Results

  • High Accuracy: OrbNet-Equi consistently outperforms standard ML methods and traditional semi-empirical QM methods on various benchmarks, achieving accuracy competitive with state-of-the-art DFT methods for many properties.
  • Significant Speedup: The method is several orders of magnitude faster than DFT, with inference times comparable to semi-empirical QM methods. For example, it is up to thousand-fold faster than composite DFT methods.
  • Improved Data Efficiency: OrbNet-Equi models show significantly better performance with fewer training samples compared to Atomistic ML methods.
  • Broad Applicability: The model accurately predicts a wide spectrum of properties, including electronic energies, dipole moments, electron densities, frontier orbital energies, reaction barriers, and non-covalent interactions.
  • Performance on Challenging Systems: OrbNet-Equi demonstrates capability in handling challenging systems like charge-transfer complexes and open-shell systems.
  • Transferability: OrbNet-Equi/SDC21, trained on small molecules, shows strong performance on diverse downstream benchmarks, including conformer ranking, torsion profiles, and non-covalent interactions.

Limitations and Future Work

  • Dependence on QM Featurization: The accuracy of OrbNet-Equi is still dependent on the quality of the underlying QM featurization (GFN-xTB in this case). Errors or limitations in the tight-binding model can propagate to the ML predictions.
  • Extrapolation Challenges: While showing improved transferability, the paper acknowledges that performance can degrade on highly extrapolative subsets where chemical compositions or physical interactions are entirely unseen during training.
  • Future Work: The authors suggest several promising directions:
  • Improving Chemical Space Coverage: Systematically adding simple model systems with absent physical interactions to the training data to enhance accuracy and coverage.
  • Developing Better QM Featurizers: Incorporating more advanced physics-based approximations into the QM featurizer to complement the ML model.
  • Extending to Material Properties: Applying the framework to material science problems.
  • Hybrid QM/ML for Complex Systems: Facilitating applications in areas like flow batteries, transition-metal catalysis, and biochemical processes using multi-scale strategies.

Summary updated

Abstract

Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. However, existing machine learning techniques are challenged by the scarcity of training data when exploring unknown chemical spaces. We overcome this barrier by systematically incorporating knowledge of molecular electronic structure into deep learning. By developing a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simulations and learned mappings to recover high-fidelity physical quantities. OrbNet-Equi accurately models a wide spectrum of target properties while being several orders of magnitude faster than density functional theory. Despite only using training samples collected from readily available small-molecule libraries, OrbNet-Equi outperforms traditional semiempirical and machine learning-based methods on comprehensive downstream benchmarks that encompass diverse main-group chemical processes. Our method also describes interactions in challenging charge-transfer complexes and open-shell systems. We anticipate that the strategy presented here will help to expand opportunities for studies in chemistry and materials science, where the acquisition of experimental or reference training data is costly.
BibTeX
@inproceedings{qiao2022informing,
  author = {Zhuoran Qiao and Anders S. Christensen and Matthew Welborn and Frederick R. Manby and Anima Anandkumar and Thomas F. Miller},
  title = {{Informing geometric deep learning with electronic interactions to accelerate quantum chemistry}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2022},
  doi = {10.1073/pnas.2205221119},
}
Figure