Critical Points in Quantum Generative Models

Eric R. Anschuetz

Abstract

One of the most important properties of classical neural networks is the clustering of local minima of the network near the global minimum, enabling efficient training. This has been observed not only numerically, but also has begun to be analytically understood through the lens of random matrix theory. Inspired by these results in classical machine learning, we show that a certain randomized class of variational quantum algorithms can be mapped to Wishart random fields on the hypertorus. Then, using the statistical properties of such random processes, we analytically find the expected distribution of critical points. Unlike the case for deep neural networks, we show the existence of a transition in the quality of local minima at a number of parameters exponentially large in the problem size. Below this transition, all local minima are concentrated far from the global minimum; above, all local minima are concentrated near the global minimum. This is consistent with previously observed numerical results on the landscape behavior of Hamiltonian agnostic variational quantum algorithms. We give a heuristic explanation as to why ansatzes that depend on the problem Hamiltonian might not suffer from these scaling issues. We also verify that our analytic results hold experimentally even at modest system sizes.

Introduction

In classical deep learning, gradient descent often succeeds despite the complex, non-convex nature of the loss landscape. This success is attributed to a high-dimensional property where most local minima cluster near the global minimum. Variational Quantum Algorithms (VQAs) were expected to inherit this property, yet practitioners frequently encounter "bad" local minima that impede training. This research investigates the geometry of the quantum loss landscape to identify the conditions under which a quantum model transitions from being untrainable to trainable.

Theoretical Framework: Mapping Quantum Landscapes

The study employs a sophisticated mathematical mapping to analyze quantum circuits. By modeling "Hamiltonian agnostic" quantum models—those whose structure is independent of the specific problem—as Wishart Hypertoroidal Random Fields (WHRFs), the authors leverage tools from differential topology and statistics. Specifically, the Kac-Rice Formula and Morse Theory are used to count critical points (minima, maxima, and saddles), while Free Probability Theory facilitates the analysis of the Hessian's eigenvalue distribution in the limit of many parameters.

Key Insights and the Phase Transition

The central discovery is a "trainability phase transition" occurring at a parameter threshold $p \approx 2m$, where $p$ is the number of parameters and $m$ is the effective dimension of the Hilbert space.

  • Sub-critical Regime ($p < 2m$): Local minima are exponentially concentrated at energy levels far from the global minimum, trapping optimizers in poor solutions.
  • Super-critical Regime ($p \geq 2m$): Local minima cluster near the global minimum, facilitating successful training.

For generic hardware-efficient circuits, $m$ scales exponentially with the number of qubits ($n$). Consequently, ensuring a "good" landscape requires an exponential number of parameters, rendering these agnostic models impractical for large-scale applications. In contrast, "Hamiltonian informed" models likely succeed by effectively reducing the dimension $m$.

Numerical Validation

The analytical proof is supported by numerical simulations of quantum circuits designed to find the ground state of the 1D spinless Fermi-Hubbard Hamiltonian. Simulations on systems up to 8 qubits ($2^n = 256$ dimensions) confirm that the actual distribution of local minima closely matches the theoretical predictions.

These results suggest that the "barren plateau" problem—where gradients vanish—is physically linked to this concentration of poor local minima. The landscape effectively becomes a flat plain populated by shallow traps.

Conclusion and Future Work

This work provides the first analytical proof of the scaling requirements for trainable quantum landscapes in agnostic models. While it highlights the limitations of generic hardware-efficient ansatze, it also provides a theoretical basis for why physics-informed models perform better. Future research will aim to extend these rigorous proofs to Hamiltonian-informed models and investigate the impact of hardware noise on landscape geometry.

Summary updated

Abstract

One of the most important properties of neural networks is the clustering of local minima of the loss function near the global minimum, enabling efficient training. Though generative models implemented on quantum computers are known to be more expressive than their traditional counterparts, it has empirically been observed that these models experience a transition in the quality of their local minima. Namely, below some critical number of parameters, all local minima are far from the global minimum in function value; above this critical parameter count, all local minima are good approximators of the global minimum. Furthermore, for a certain class of quantum generative models, this transition has empirically been observed to occur at parameter counts exponentially large in the problem size, meaning practical training of these models is out of reach. Here, we give the first proof of this transition in trainability, specializing to this latter class of quantum generative model. We use techniques inspired by those used to study the loss landscapes of classical neural networks. We also verify that our analytic results hold experimentally even at modest model sizes.
BibTeX
@inproceedings{anschuetz2022critical,
  author = {Eric R. Anschuetz},
  title = {{Critical Points in Quantum Generative Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=2f1z55GVQN},
}
Figure