Sampling-free Inference for Ab-Initio Potential Energy Surface Networks

Nicholas Gao, Stephan Günnemann

Abstract

Recently, it has been shown that neural networks not only approximate the ground-state wave functions of a single molecular system well but can also generalize to multiple geometries. While such generalization significantly speeds up training, each energy evaluation still requires Monte Carlo integration which limits the evaluation to a few geometries. In this work, we address the inference shortcomings by proposing the Potential learning from ab-initio Networks (PlaNet) framework, in which we simultaneously train a surrogate model in addition to the neural wave function. At inference time, the surrogate avoids expensive Monte-Carlo integration by directly estimating the energy, accelerating the process from hours to milliseconds. In this way, we can accurately model high-resolution multi-dimensional energy surfaces for larger systems that previously were unobtainable via neural wave functions. Finally, we explore an additional inductive bias by introducing physically-motivated restricted neural wave function models. We implement such a function with several additional improvements in the new PESNet++ model. In our experimental evaluation, PlaNet accelerates inference by 7 orders of magnitude for larger molecules like ethanol while preserving accuracy. Compared to previous energy surface networks, PESNet++ reduces energy errors by up to 74%.

Introduction

Solving the Schrödinger equation from first principles (ab-initio) is a cornerstone of quantum chemistry, enabling the prediction of molecular behavior and properties. Recent advancements have leveraged neural networks within the Variational Monte Carlo (VMC) framework to approximate these solutions with high precision. However, a significant bottleneck remains: inference speed. Calculating the energy for a single molecular geometry typically requires thousands of Monte Carlo samples, a process that can take hours. This computational cost prohibits the use of neural wave functions in large-scale applications like molecular dynamics or high-throughput geometry optimization.

The PlaNet Framework

To overcome this limitation, we introduce PlaNet (Potential learning from ab-initio Networks). The core innovation of PlaNet is the simultaneous training of a fast surrogate model alongside the physics-based neural wave function.

The PlaNet framework architecture showing the training of the surrogate model alongside the VMC process.

Instead of discarding the noisy intermediate energy estimates generated during VMC training, PlaNet "recycles" this data to train a Graph Neural Network (DimeNet++) as a surrogate. By employing an adaptive Exponential Moving Average (EMA) to weight the surrogate’s parameters, the model effectively filters out early-stage noise and converges toward the highly accurate late-stage energy estimates. This online learning approach provides a high-speed inference model as a direct byproduct of the training process.

PESNet++: Architectural Enhancements

In addition to the PlaNet framework, we present PESNet++, an improved neural wave function architecture. PESNet++ incorporates several key physical and architectural inductive biases:

  • Restricted Neural Wave Functions: For closed-shell systems, we enforce a constraint where spin-up and spin-down orbitals are identical, simplifying the learning landscape.
  • Activation and Initialization: We replaced standard activation functions with SiLU and implemented zero-bias initialization, reducing the "dead neuron" problem from over 40% to less than 10%.
  • Coordinate Transforms: A novel coordinate transform was introduced to better handle significant geometric variations during training, ensuring stability and accuracy across diverse potential energy surfaces.

Results and Evaluation

Our framework was evaluated on systems ranging from simple hydrogen chains to complex molecules like ethanol. The results demonstrate a transformative improvement in both speed and accuracy.

Comparison of energy errors for the nitrogen dimer (left) and torsion angle energy for ethanol (right).

For ethanol, PlaNet achieved a 7-order-of-magnitude speedup, reducing inference time from 2.7 hours to just 118 microseconds per geometry. Crucially, this speedup does not sacrifice accuracy; the surrogate model consistently maintains "chemical accuracy" (errors below 1.6 mEh). Furthermore, PESNet++ significantly outperformed its predecessor, reducing energy errors on the challenging nitrogen dimer ($N_2$) by 74%.

Conclusion

PlaNet and PESNet++ bridge the gap between high-accuracy ab-initio calculations and the speed required for practical chemical simulations. By recycling training data into a surrogate model and incorporating physical constraints into the wave function architecture, we enable the modeling of high-resolution energy surfaces that were previously computationally inaccessible. Future work will focus on scaling these methods to higher-dimensional surfaces and extending the restricted models to handle open-shell systems.

Summary updated

Abstract

Recently, it has been shown that neural networks not only approximate the ground-state wave functions of a single molecular system well but can also generalize to multiple geometries. While such generalization significantly speeds up training, each energy evaluation still requires Monte Carlo integration which limits the evaluation to a few geometries. In this work, we address the inference shortcomings by proposing the Potential learning from ab-initio Networks (PlaNet) framework, in which we simultaneously train a surrogate model in addition to the neural wave function. At inference time, the surrogate avoids expensive Monte-Carlo integration by directly estimating the energy, accelerating the process from hours to milliseconds. In this way, we can accurately model high-resolution multi-dimensional energy surfaces for larger systems that previously were unobtainable via neural wave functions. Finally, we explore an additional inductive bias by introducing physically-motivated restricted neural wave function models. We implement such a function with several additional improvements in the new PESNet++ model. In our experimental evaluation, PlaNet accelerates inference by 7 orders of magnitude for larger molecules like ethanol while preserving accuracy. Compared to previous energy surface networks, PESNet++ reduces energy errors by up to 74%.
BibTeX
@inproceedings{gao2023sampling,
  author = {Nicholas Gao and Stephan Günnemann},
  title = {{Sampling-free Inference for Ab-Initio Potential Energy Surface Networks}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=Tuk3Pqaizx},
}
Figure