Improved Optimization for the Neural-network Quantum States and Tests on the Chromium Dimer

Xiang Li, Jia-Cheng Huang, Guang-Ze Zhang, Hao-En Li, Zhu-Ping Shen, Chen Zhao, Jun Li, Han‐Shi HuarXiv 2024 · DOI · arXiv

TL;DR

This paper introduces three algorithmic enhancements—adaptive learning rates, constrained subspace optimization, and block-based parameter updates—to make Neural-network Quantum States (NQS) computationally efficient for chemistry. By applying these techniques, the authors successfully modeled the notoriously difficult chromium dimer ($Cr_2$), achieving higher accuracy than standard coupled-cluster methods at a significantly lower computational cost.

What Problem Does This Paper Address?

Solving the many-electron Schrödinger equation is the "holy grail" of quantum chemistry, but it is hindered by electron correlation. Standard methods like Coupled Cluster (CC) work well for simple molecules but fail when bonds are stretched or when dealing with transition metals (like Chromium), where multiple electronic states become nearly degenerate (static correlation).

Neural-network Quantum States (NQS), specifically those based on Restricted Boltzmann Machines (RBMs), offer a powerful way to represent these complex wave functions. However, optimizing these networks using Variational Monte Carlo (VMC) is notoriously slow and computationally expensive. The "Stochastic Reconfiguration" (SR) method used for optimization requires inverting large matrices, which doesn't scale well as the number of neural network parameters grows.

Methods

The authors propose a "triple threat" of algorithmic improvements to the VMC-NQS framework:

  1. Adaptive Learning Rate: Instead of guessing a fixed step size, the algorithm evaluates multiple potential learning rates in parallel. To keep this fast, they use a "truncated" energy expression that only looks at a small, deterministically selected subset of important electron configurations.
  2. Constrained Optimization within Subspace: Traditionally, VMC resamples the electron configurations after every single parameter update. The authors found that if the wave function hasn't changed much (overlap > 0.98), they can reuse the same configuration sample for multiple updates ($N_{opt}$). This acts like a "rough diagonalization" of the Hamiltonian in a local subspace, speeding up convergence.
  3. Block Optimization: To handle large RBMs, they decompose the network into smaller "blocks." By updating only one block of parameters at a time, they turn one massive, intractable matrix inversion problem into several smaller, manageable ones.
  4. Physics-Informed Initialization: Rather than starting with random noise, they initialize the network's input biases using the negative values of orbital energies (a Boltzmann distribution). This gives the model a physically sensible starting point.

Datasets

The researchers evaluated their refined algorithms on three increasingly difficult molecular systems:

  • Water ($H_2O$): Symmetric bond stretching in the 6-31G and cc-pVDZ basis sets.
  • Nitrogen ($N_2$): Dissociation curve in the cc-pVDZ basis set (a classic test for strong correlation).
  • Chromium Dimer ($Cr_2$): Ground-state energy in the Ahlrichs SV basis set. $Cr_2$ is considered one of the most challenging systems in quantum chemistry due to its complex sextuple bond and massive number of possible electron configurations ($1.6 \times 10^{22}$ in the full space).

Key Insights and Ideas

  • Subspace Stability: The most significant "aha" moment is that the configuration space doesn't need to be refreshed constantly. Reusing a "good" sample of configurations for several iterations (constrained optimization) provides a massive speedup without sacrificing accuracy.
  • RBMs are Modular: The authors mathematically demonstrated that a large RBM can be treated as a product of smaller RBMs. This justifies the "Block Optimization" strategy, allowing NQS to scale to much larger molecular systems than previously possible.
  • Better than Coupled Cluster: In regions where bonds are stretched, standard "gold standard" methods like CCSD(T) break down. The NQS approach remains robust, tracking the true energy curve much more closely.

Key Results

  • Efficiency: The adaptive learning rate and constrained optimization reduced the number of iterations and wall-clock time significantly compared to standard constant-rate VMC.
  • Accuracy on $Cr_2$: For the chromium dimer, the NQS variational energy outperformed the CCSDT method. After applying a second-order perturbative correction, the NQS results were superior to both CCSD(T) and CCSDTQ.
  • Chemical Accuracy: The NQS approach achieved results comparable to high-level (and much more expensive) methods like DMRG and Heat-bath Configuration Interaction (HCI), but at a "modest CPU cost."

Limitations and Future Work

While RBMs are powerful, the authors acknowledge they still have expressive limits when trying to capture the full complexity of very large molecules with intricate electronic structures.

Future directions include: * Applying these optimization strategies to more advanced "deep" architectures like FermiNet or Autoregressive Neural Networks. * Exploring even larger active spaces where traditional deterministic selection of configurations becomes impossible. * Further refining the perturbative corrections to bridge the gap between variational NQS results and the exact Full Configuration Interaction (FCI) limit.

Summary updated

Abstract

The advent of Neural-network Quantum States (NQS) has significantly advanced wave function ansatz research, sparking a resurgence in orbital space variational Monte Carlo (VMC) exploration. This work introduces three algorithmic enhancements to reduce computational demands of VMC optimization using NQS: an adaptive learning rate algorithm, constrained optimization, and block optimization. We evaluate the refined algorithm on complex multireference bond stretches of $\rm H_2O$ and $\rm N_2$ within the cc-pVDZ basis set and calculate the ground-state energy of the strongly correlated chromium dimer ($\rm Cr_2$) in the Ahlrichs SV basis set. Our results achieve superior accuracy compared to coupled cluster theory at a relatively modest CPU cost. This work demonstrates how to enhance optimization efficiency and robustness using these strategies, opening a new path to optimize large-scale Restricted Boltzmann Machine (RBM)-based NQS more effectively and marking a substantial advancement in NQS's practical quantum chemistry applications.
BibTeX
@misc{li2024improved,
  author = {Xiang Li and Jia-Cheng Huang and Guang-Ze Zhang and Hao-En Li and Zhu-Ping Shen and Chen Zhao and Jun Li and Han‐Shi Hu},
  title = {{Improved Optimization for the Neural-network Quantum States and Tests on the Chromium Dimer}},
  year = {2024},
  eprint = {2404.09280},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.09280},
}
Figure