Report on scipost_202112_00047v1

Vicentini Filippo, Damian ⋆, Attila Szabó, Dian Wu, Christopher Roth, Clemens Giuliani, Gabriel Pescia, Jannes Nys, Vladimir Vargas-Calderón, Nikita Astrakhantsev, Giuseppe CarleoOpenalex Unknown 2022 · DOI · OpenAlex

Abstract

We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural-network quantum states (NQS) and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network ansätze in pure Python code using the concise notation of machine learning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.

Introduction: The Many-Body Challenge

Solving the many-body Schrödinger equation remains one of the most formidable challenges in modern physics. The primary obstacle is the exponential growth of the quantum wave function's complexity as the number of particles increases. Traditional computational methods often struggle with this "curse of dimensionality." Variational methods, particularly Neural-Network Quantum States (NQS), have emerged as a powerful alternative. By using neural networks as efficient compressors for high-dimensional wave functions, researchers can approximate the state of complex quantum systems with high precision.

However, standard machine learning frameworks like PyTorch or TensorFlow are not natively designed for the nuances of quantum mechanics. Quantum simulations require complex-valued amplitudes, specific Hilbert space constraints, and specialized optimization techniques like the calculation of the Quantum Geometric Tensor (QGT). NetKet 3 addresses these gaps, providing a flexible, high-performance bridge between modern differentiable programming and quantum physics.

Technical Core: The JAX Integration

The most significant evolution in NetKet 3 is its transition to a JAX-based backend. This shift from a custom C++ core to a pure Python environment—without sacrificing performance—enables several key technical advancements:

  • Pure Python Ansätze: Users can now define custom neural network architectures using frameworks like Flax or Haiku. JAX’s jit compiler transforms this Python code into optimized machine code via XLA (Accelerated Linear Algebra).
  • Automatic Differentiation (AD): By leveraging JAX's AD capabilities, gradients of quantum observables are generated automatically. This eliminates the need for manual derivation of complex backpropagation rules for quantum operators.
  • Multiple Dispatch: Utilizing the plum module, NetKet 3 employs a multiple dispatch system. This allows the library to automatically select the most efficient algorithm for calculating expectation values based on the specific combination of the variational state and the operator involved.
  • Efficient Optimization: For natural gradient descent, also known as Stochastic Reconfiguration in the physics community, NetKet 3 offers specialized implementations. QGTJacobian provides high speed for smaller systems, while QGTOnTheFly offers memory efficiency that scales to models with millions of parameters.

Enforcing Physical Constraints

A major insight in NetKet 3 is the treatment of physical symmetries. Rather than hoping a network learns the symmetries of a crystal lattice, NetKet 3 allows researchers to bake these constraints directly into the architecture using Group Convolutional Neural Networks (GCNNs). By enforcing symmetries like translations or rotations, the variational search is restricted to the correct physical sector, significantly reducing the parameter space and accelerating convergence.

Furthermore, NetKet 3 models the logarithm of the wave function, $\ln \psi(s)$. This approach is critical because quantum amplitudes can span many orders of magnitude. Working in log-space stabilizes the optimization landscape, making it compatible with standard machine learning optimizers.

Performance and Versatility

The framework has been benchmarked against a variety of challenging physical models, including the 1D/2D Transverse-Field Ising Model (TFIM) and the frustrated $J_1-J_2$ Heisenberg model on a $10 \times 10$ square lattice. Key results include:

  • Computational Speed: NetKet 3 outperforms comparable JAX-based libraries by nearly an order of magnitude on multi-core CPUs when using SVD-based solvers. On GPUs, iterative solvers achieve massive speedups, reducing the time for Variational Monte Carlo (VMC) steps from minutes to seconds.
  • Scalability: Through integration with mpi4jax, the framework demonstrates near-linear scaling across multiple computing nodes, enabling the simulation of systems with thousands of degrees of freedom.
  • Broad Application: Beyond ground-state searches, the unified interface supports excited states, real-time unitary dynamics, and dissipative open-system dynamics via the Lindblad master equation.

Limitations and Future Directions

While NetKet 3 represents a significant leap forward, development continues in several areas. Native, high-performance support for fermionic systems—which require anti-symmetric wave functions—is currently under active development. Additionally, while the framework supports TPUs, numerical instabilities can arise due to the current prevalence of half-precision (float16) on those accelerators. Future updates will also focus on moving more quantum primitive operations, such as operator indexing, directly onto the GPU to eliminate CPU bottlenecks in smaller systems.

NetKet 3 stands as a modular, "Lego-like" ecosystem. Whether researchers need the full suite for a complex simulation or just the MPI-parallelized Monte Carlo samplers for a custom solver, the library provides the necessary tools to advance the frontier of computational quantum physics.

Summary updated

Abstract

We introduce version 3 of NetKet, the machine learning toolbox for manybody quantum physics. NetKet is built around neural-network quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network anstze in pure Python code using the concise notation of machinelearning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.
BibTeX
@inproceedings{filippo2022report,
  author = {Vicentini Filippo and Damian ⋆ and Attila Szabó and Dian Wu and Christopher Roth and Clemens Giuliani and Gabriel Pescia and Jannes Nys and Vladimir Vargas-Calderón and Nikita Astrakhantsev and Giuseppe Carleo},
  title = {{Report on scipost\_202112\_00047v1}},
  booktitle = {Openalex Unknown},
  year = {2022},
  doi = {10.21468/scipost.report.4376},
}
Figure