Autoregressive Neural TensorNet: Bridging Neural Networks and Tensor Networks for Quantum Many-Body Simulation

Zhuo Chen, Laker Newhouse, Eddie Chen, Di Luo, Marin Soljacic

Abstract

Quantum many-body physics simulation has important impacts on understanding fundamental science and has applications to quantum materials design and quantum technology. However, due to the exponentially growing size of the Hilbert space with respect to the particle number, a direct simulation is intractable. While representing quantum states with tensor networks and neural networks are the two state-of-the-art methods for approximate simulations, each has its own limitations in terms of expressivity and inductive bias. To address these challenges, we develop a novel architecture, Autoregressive Neural TensorNet (ANTN), which bridges tensor networks and autoregressive neural networks. We show that Autoregressive Neural TensorNet parameterizes normalized wavefunctions, allows for exact sampling, generalizes the expressivity of tensor networks and autoregressive neural networks, and inherits a variety of symmetries from autoregressive neural networks. We demonstrate our approach on quantum state learning as well as finding the ground state of the challenging 2D $J_1$-$J_2$ Heisenberg model with different systems sizes and coupling parameters, outperforming both tensor networks and autoregressive neural networks. Our work opens up new opportunities for quantum many-body physics simulation, quantum technology design, and generative modeling in artificial intelligence.

Introduction: The Challenge of Quantum Complexity

Simulating quantum many-body systems is a cornerstone of modern physics, yet it faces the "Curse of Dimensionality." As the number of particles in a system increases, the Hilbert space required to describe its state grows exponentially. For instance, a system with 300 qubits possesses more potential states than there are atoms in the observable universe, making direct simulation impossible.

To navigate this complexity, researchers employ "ansatzes"—mathematical representations that approximate quantum states. Currently, two paradigms dominate:

  1. Tensor Networks (TNs): These models, such as Matrix Product States (MPS), are built on physical principles like the area law of entanglement. While physically intuitive, they often struggle with high-dimensional systems (2D or 3D) and can be extremely memory-intensive.
  2. Neural Network Quantum States (NNQS): Leveraging the flexibility of deep learning, these models can learn complex patterns. However, they often lack "physics priors" and struggle with the sign problem—the difficulty of learning the fluctuating sign structure of complex-valued quantum wavefunctions.

The Autoregressive Neural TensorNet (ANTN) is proposed as a bridge between these two methodologies, combining the physical inductive bias of tensor networks with the expressive power of neural networks.

Methodology: The ANTN Architecture

ANTN integrates the strengths of Autoregressive Neural Networks (ARNNs) and Tensor Networks. Instead of treating them as separate entities, ANTN uses an ARNN (such as a Transformer or PixelCNN) to generate the constituent tensors of a Matrix Product State.

The ANTN architecture bridges Autoregressive Neural Networks (ARNN) and Tensor Networks (TN), combining high expressivity with physical inductive bias.

Key technical features of the ANTN framework include:

  • Hybrid Structure: By generating TN components via a neural network, ANTN can represent states with "volume law" entanglement, surpassing the "area law" limitations of traditional MPS with fixed bond dimensions.
  • Exact Sampling: Unlike many neural methods that rely on potentially unstable Markov Chain Monte Carlo (MCMC) sampling, ANTN supports exact, sequential sampling. This ensures faster and more reliable calculation of physical observables.
  • Symmetry Inheritance: ANTN naturally inherits physical symmetries (e.g., spin-flip or particle number conservation) from its underlying ARNN, constraining the model to physically valid regions of the Hilbert space.
  • Physics-Informed Initialization: The model can be pre-trained using classical methods like the Density Matrix Renormalization Group (DMRG), providing a physically grounded starting point for further neural optimization.

Benchmarking and Results

The performance of ANTN was evaluated against several challenging benchmarks, including Random Bell States, Shallow Random Circuits, and the 2D $J_1-J_2$ Heisenberg model.

Overcoming the Sign Problem and Entanglement

A critical test for any quantum ansatz is its ability to handle complex sign structures and high entanglement. In tests involving Random Bell States (high entanglement) and Shallow Random Circuits (complex signs), ANTN demonstrated superior robustness.

Fidelity comparison on Random Bell States and Shallow Random Circuits. ANTN maintains high performance even without explicit sign rules, outperforming standard ARNNs and low-bond-dimension MPS.

As shown in Figure 4, standard ARNNs suffer a significant performance drop when the "sign rule" is not explicitly provided. In contrast, ANTN maintains high fidelity, effectively "discovering" the underlying physics. Furthermore, ANTN achieves near-perfect fidelity on Bell states where traditional MPS with low bond dimensions fail, showcasing its ability to represent volume-law entanglement.

State-of-the-Art Performance on the $J_1-J_2$ Model

The 2D $J_1-J_2$ Heisenberg model is a "gold standard" challenge in condensed matter physics due to its competing interactions and complex quantum phases. In simulations of 12x12 systems, the "elementwise" ANTN variant outperformed both pure PixelCNNs and massive Matrix Product States (bond dimension 1024). Remarkably, ANTN achieved these superior results while using only approximately 1% of the parameters required by the traditional MPS approach.

Conclusion and Future Directions

The Autoregressive Neural TensorNet represents a significant advancement in quantum simulation. By bridging the gap between TNs and ARNNs, it provides a parameter-efficient, expressive, and physically grounded framework for studying many-body systems.

While the current work focuses on Matrix Product States, future iterations could integrate more complex 2D tensor networks like Projected Entangled Pair States (PEPS). Beyond physics, the structural inductive biases of ANTN offer potential benefits for general generative modeling in artificial intelligence, where capturing complex dependencies is crucial.

Summary updated

Abstract

Quantum many-body physics simulation has important impacts on understanding fundamental science and has applications to quantum materials design and quantum technology. However, due to the exponentially growing size of the Hilbert space with respect to the particle number, a direct simulation is intractable. While representing quantum states with tensor networks and neural networks are the two state-of-the-art methods for approximate simulations, each has its own limitations in terms of expressivity and inductive bias. To address these challenges, we develop a novel architecture, Autoregressive Neural TensorNet (ANTN), which bridges tensor networks and autoregressive neural networks. We show that Autoregressive Neural TensorNet parameterizes normalized wavefunctions, allows for exact sampling, generalizes the expressivity of tensor networks and autoregressive neural networks, and inherits a variety of symmetries from autoregressive neural networks. We demonstrate our approach on quantum state learning as well as finding the ground state of the challenging 2D $J_1$-$J_2$ Heisenberg model with different systems sizes and coupling parameters, outperforming both tensor networks and autoregressive neural networks. Our work opens up new opportunities for quantum many-body physics simulation, quantum technology design, and generative modeling in artificial intelligence.
BibTeX
@inproceedings{chen2023autoregressive,
  author = {Zhuo Chen and Laker Newhouse and Eddie Chen and Di Luo and Marin Soljacic},
  title = {{Autoregressive Neural TensorNet: Bridging Neural Networks and Tensor Networks for Quantum Many-Body Simulation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=qdsDy0zbn4},
}
Figure