Sampling-free Inference for Ab-Initio Potential Energy Surface Networks

Nicholas Gao, Stephan Günnemann

In quantum chemistry, solving the Schrödinger equation ab-initio (from first principles, without empirical data) is the gold standard for understanding molecular behavior. Recently, neural networks have emerged as powerful tools for approximating these solutions using a framework called Variational Monte Carlo (VMC).

However, this approach suffers from a massive computational bottleneck: inference is painfully slow. To calculate the energy of a single molecular geometry, these models must perform expensive Monte Carlo integration, which scales poorly ($O(N^4)$) with the number of electrons. Simulating a molecule moving (Molecular Dynamics) or finding its most stable shape (Geometry Optimization) requires hundreds of thousands of these evaluations. With current neural methods taking hours per step, they remain largely impractical for complex chemistry.

To address this, researchers have developed PlaNet (Potential learning from ab-initio Networks), a framework that bypasses the computational tax of Monte Carlo integration. By training a lightning-fast surrogate model alongside a neural wave function, PlaNet achieves a 7-order-of-magnitude speedup in molecular energy calculations (from hours to milliseconds) while simultaneously improving accuracy by up to 74% through architectural refinements.

The PlaNet Framework and PESNet++

The authors introduce two major innovations to solve the inference bottleneck and improve model accuracy:

1. PlaNet: A "Trash-to-Treasure" Surrogate

Usually, during the training of a neural wave function, the model generates "noisy" local energy estimates that are used for optimization and then discarded. PlaNet proposes a clever "trash-to-treasure" approach: it uses these intermediate noisy energies as training labels for a separate surrogate model (a Graph Neural Network called DimeNet++).

The PlaNet framework architecture.

Because the surrogate is trained online during the main optimization, it learns the smooth potential energy surface essentially "for free." One might assume noisy intermediate energies are bad for training, but because the VMC process generates so many samples, the surrogate GNN effectively averages out the noise, recovering a smooth, accurate energy surface. At inference time, researchers can query the surrogate directly, skipping the Monte Carlo integration entirely.

2. PESNet++: Upgrading the Wave Function

Alongside PlaNet, the authors introduce an upgraded architecture for the underlying wave function called PESNet++, which features:

  • Restricted Neural Wave Functions: For "closed-shell" systems (where electrons are paired), it forces spin-up and spin-down orbitals to be identical, mimicking classical Restricted Hartree-Fock theory. This inductive bias reduces the parameter space, making the optimization landscape much easier for the neural network to navigate.
  • Dense Orbitals: Moving from restricted block-diagonal matrices to dense ones to capture more complex electron interactions.
  • Coordinate Transform: A trick that shifts electron positions along with nuclei during training. This acts as an "anchor," preventing the decorrelation that usually happens when a molecule's geometry changes significantly, allowing the model to learn across a wider range of shapes without MCMC sampling failures.

Key Results and Chemical Accuracy

The authors evaluated their framework on several benchmark molecular systems, ranging from small clusters ($H_4$, $H_{10}$) and difficult diatomic molecules ($N_2$) to complex 6D potential energy surfaces ($H_2-HF$) and larger organic molecules like ethanol ($C_2H_5OH$).

Extreme Speedup: For larger molecules like ethanol, PlaNet accelerates inference by 7 orders of magnitude. What previously took ~2.7 hours per geometry evaluation now takes approximately 0.11 milliseconds.

Torsion energy curve for ethanol.

Superior Accuracy: The speedup does not come at the cost of physical reliability. The surrogate model consistently achieved "chemical accuracy" (errors below 1.6 mEh) across various systems. Furthermore, the architectural improvements in PESNet++ reduced energy errors on the notoriously challenging $N_2$ molecule by 74% compared to the previous state-of-the-art PESNet.

Limitations and Future Work

While PlaNet represents a massive leap forward for neural quantum chemistry, some limitations remain:

  • The Curse of Dimensionality: While the 6D surface of $H_2-HF$ was modeled well, as the number of atoms grows, the energy surface becomes exponentially harder to sample and fit.
  • Singlets Only: The "Restricted" wave function improvement is specifically designed for singlet states (no unpaired electrons). It cannot properly describe radicals or systems with multiple unpaired electrons, such as $O_2$.
  • Surrogate Expressivity: The chosen surrogate (DimeNet++) has known limitations in distinguishing certain molecular symmetries. Future work could integrate more advanced equivariant architectures to improve generalization further.

Summary updated

Abstract

Recently, it has been shown that neural networks not only approximate the ground-state wave functions of a single molecular system well but can also generalize to multiple geometries. While such generalization significantly speeds up training, each energy evaluation still requires Monte Carlo integration which limits the evaluation to a few geometries. In this work, we address the inference shortcomings by proposing the Potential learning from ab-initio Networks (PlaNet) framework, in which we simultaneously train a surrogate model in addition to the neural wave function. At inference time, the surrogate avoids expensive Monte-Carlo integration by directly estimating the energy, accelerating the process from hours to milliseconds. In this way, we can accurately model high-resolution multi-dimensional energy surfaces for larger systems that previously were unobtainable via neural wave functions. Finally, we explore an additional inductive bias by introducing physically-motivated restricted neural wave function models. We implement such a function with several additional improvements in the new PESNet++ model. In our experimental evaluation, PlaNet accelerates inference by 7 orders of magnitude for larger molecules like ethanol while preserving accuracy. Compared to previous energy surface networks, PESNet++ reduces energy errors by up to 74%.
BibTeX
@inproceedings{gao2023sampling,
  author = {Nicholas Gao and Stephan Günnemann},
  title = {{Sampling-free Inference for Ab-Initio Potential Energy Surface Networks}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=Tuk3Pqaizx},
}
Figure