Abstract
Modern deep learning has enabled unprecedented achievements in various domains. Nonetheless, employment of machine learning for wave function representations is focused on more traditional architectures such as restricted Boltzmann machines (RBMs) and fully-connected neural networks. In this letter, we establish that contemporary deep learning architectures, in the form of deep convolutional and recurrent networks, can efficiently represent highly entangled quantum systems. By constructing Tensor Network equivalents of these architectures, we identify an inherent reuse of information in the network operation as a key trait which distinguishes them from standard Tensor Network based representations, and which enhances their entanglement capacity. Our results show that such architectures can support volume-law entanglement scaling, polynomially more efficiently than presently employed RBMs. Thus, beyond a quantification of the entanglement capacity of leading deep learning architectures, our analysis formally motivates a shift of trending neural-network based wave function representations closer to the state-of-the-art in machine learning.
Introduction
Simulating many-body quantum systems is one of the most significant challenges in modern physics, primarily due to the exponential growth of quantum entanglement complexity with the number of particles. While Tensor Networks (TNs), such as Matrix Product States (MPS), have proven successful for one-dimensional systems obeying the "area-law" of entanglement, they often falter when faced with two-dimensional systems or highly entangled "volume-law" states.
Recently, the physics community has turned to machine learning, utilizing architectures like Restricted Boltzmann Machines (RBMs) to represent quantum wave functions. However, these are relatively dated architectures compared to the state-of-the-art in deep learning. This research establishes a formal bridge between modern deep learning architectures—specifically deep Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs)—and many-body quantum physics, demonstrating their superior capacity for representing complex entanglement.
Theoretical Framework: Mapping Neural Networks to Tensor Networks
The authors bridge the gap between machine learning and physics by mapping specific neural network architectures to Tensor Networks. They focus on Arithmetic Circuits: Convolutional Arithmetic Circuits (CACs) and Recurrent Arithmetic Circuits (RACs). These are specialized versions of CNNs and RNNs where non-linearities are restricted to products, making them mathematically equivalent to specific tensor decompositions.
A critical structural difference identified is "information re-use." While standard TNs are typically tree-like or linear, modern CNNs utilize overlapping kernels, and RNNs employ stacked layers. In the language of Tensor Networks, this corresponds to "duplicating" or re-using internal data, a feature that significantly enhances expressive power.
The Power of Overlap and Depth
The analysis reveals that the simple act of allowing kernels to overlap (where the stride is less than the kernel size) exponentially increases a network's expressive power. This overlap creates an "information re-use" that standard, non-redundant TNs cannot easily replicate. This mechanism allows overlapping CNNs to bypass the limitations of traditional tree-structured TNs and support volume-law entanglement scaling in 2D systems—the most difficult states to simulate classically.
Similarly, deep (multi-layer) RNNs are shown to support logarithmic corrections to the area-law in 1D. This makes them qualitatively similar to the MERA (Multi-scale Entanglement Renormalization Ansatz), a powerful but computationally expensive TN used for critical quantum systems.
Key Results and Efficiency Gains
The study provides formal proofs for the entanglement capacity of these architectures:
- Efficiency: To represent volume-law entanglement in a 2D system of $N$ particles, a fully-connected network requires $O(N^2)$ parameters and an RBM requires $O(N)$. In contrast, an overlapping CNN requires only $O(\sqrt{N})$ parameters.
- CNN Scaling (Theorem 1): Overlapping CNNs support entanglement entropy scaling with the volume ($\alpha^d$) for systems of linear size $\alpha$, provided the network depth $L > \alpha/K$.
- RNN Scaling (Theorem 2): Deep recurrent networks support a maximal entanglement entropy that scales as $\Omega(\log |A|)$, where $|A|$ is the subsystem size, matching the behavior of critical quantum systems at phase transitions.
- Computational Advantage: The cost of computing wave function amplitudes with these CNNs is linear in the system size, offering a significant practical advantage over the often quadratic cost associated with RBMs.
Conclusion and Future Outlook
This work formally motivates a shift in neural-network-based wave function representations toward state-of-the-art deep learning architectures. While the proofs rely on Arithmetic Circuits with product pooling, they provide a rigorous foundation for understanding the entanglement capacity of more standard non-linear networks. Future work will need to address optimization challenges, as proving expressivity does not guarantee that standard training algorithms like Variational Monte Carlo will easily locate these highly entangled states within the parameter space. Nonetheless, the theoretical efficiency of CNNs and RNNs marks a promising path forward for investigating complex 2D many-body systems.
Summary updated