Identifying quantum phases via disentanglement based on deep reinforcement learning

Zheng An, Chenfeng Cao, Dongruo ZhouarXiv 2021 · arXiv

Abstract: Identifying phases of matter is a complex task in quantum theory, where ground-state complexity grows exponentially with system size. Traditionally, this required physicists to identify specific order parameters for each phase. This work proposes a radically different approach: using deep reinforcement learning (RL) to develop efficient variational quantum circuits that disentangle the ground states of many-body systems. By analyzing how specific circuit structures reduce entanglement entropy across a phase diagram, the researchers can identify quantum phase transitions and discover underlying physical symmetries without a priori knowledge of order parameters.

Introduction

The identification of quantum phases of matter is a cornerstone of condensed matter physics. While traditional methods rely on order parameters—physical quantities like magnetization that change abruptly at a transition—many complex systems, particularly those with topological order, lack obvious indicators. Quantum entanglement offers a universal "fingerprint" for these phases, but its calculation for large systems is computationally prohibitive. This paper presents a clever intersection of many-body physics and machine learning to address this challenge.

The Disentanglement Framework

The core idea is single-qubit disentanglement. The researchers hypothesized that the ground state of a system within a specific phase possesses a characteristic entanglement structure. A quantum circuit optimized to "untangle" a single qubit from its neighbors in one phase will perform differently when applied to a state in a different phase.

Overview of the disentanglement-based phase identification process showing how circuits trained on different phases can pinpoint critical points.

By measuring the local entanglement entropy of a target qubit after it passes through a specialized circuit, researchers can pinpoint the exact location of quantum critical points. The intersection of performance curves from circuits trained on different phases (e.g., Phase A and Phase B) marks the transition point.

Reinforcement Learning for Circuit Design

To find these optimal "disentanglers," the team employed a Deep Q-Learning (DQN) agent. Unlike traditional variational approaches that use fixed circuit templates, the RL agent builds the circuit gate-by-gate, selecting from a universal set of operations including rotations, Hadamard, and CNOT gates.

The deep reinforcement learning framework (a) and the layer-by-layer circuit construction process (b) used to design the disentanglement circuits.

The agent's goal is to minimize the local entanglement entropy of a target qubit. Once the RL agent determines the optimal circuit structure (the arrangement of gates), a classical optimizer (BFGS) fine-tunes the rotation angles. This hybrid approach allows for the discovery of compact, efficient circuits that are tailored to the specific entanglement properties of the phase.

Key Insights and Results

The method was evaluated on two paradigmatic 1D models: the Transverse Field Ising Model (TFIM) and the XXZ model. The results yielded several striking insights:

  • Structural Fingerprints: States within the same phase share a similar "disentangling architecture." A circuit trained at one point in a phase generalizes remarkably well to all other states in that same phase.
  • Learning Duality: In the TFIM, the RL agent effectively "learned" the Kramers-Wannier duality. The agent discovered that the optimal circuit for the ferromagnetic phase was mathematically related to the circuit for the paramagnetic phase through a duality transformation—a deep symmetry of the model discovered purely through trial and error.
  • Scalability and Robustness: Circuits trained on small 8-qubit systems remained accurate when applied to systems of up to 14 qubits. Furthermore, the method proved robust against "noisy" ground states generated by Variational Quantum Eigensolvers (VQE), making it suitable for near-term (NISQ) quantum hardware.

Conclusion

This study demonstrates that reinforcement learning can effectively map quantum phases by learning the local entanglement structures of ground states. By focusing on single-qubit disentanglement and local measurements, the method offers a scalable and efficient alternative to traditional classification techniques. This approach is uniquely suited for detecting complex phases where traditional order parameters fail, potentially opening new doors for studying topological order and other exotic many-body phenomena.

Reference: An, Z., Cao, C., Xu, C. Q., & Zhou, D. L. (2024). Learning quantum phases via single-qubit disentanglement. Quantum, 8, 1402. arXiv:2107.03542v4

Summary updated

Abstract

Identifying phases of matter is a complicated process, especially in quantum theory, where the complexity of the ground state appears to rise exponentially with system size. Traditionally, physicists have been responsible for identifying suitable order parameters for the identification of the various phases. The entanglement of quantum many-body systems exhibits rich structures and can be determined over the phase diagram. The intriguing question of the relationship between entanglement and quantum phase transition has recently been addressed. As a method that works directly on the entanglement structure, disentanglement can provide factual information on the entanglement structure. In this work, we follow a radically different approach to identifying quantum phases: we utilize reinforcement learning (RL) approaches to develop an efficient variational quantum circuit that disentangles the ground-state of Ising spin chain systems. We show that the specified quantum circuit structure of the tested models correlates to a phase transition in the behaviour of the entanglement. We show a similar universal quantum circuit structure for ground states in the same phase to reduce entanglement entropy. This study sheds light on characterizing quantum phases with the types of entanglement structures of the ground states.
BibTeX
@misc{an2021identifying,
  author = {Zheng An and Chenfeng Cao and Dongruo Zhou},
  title = {{Identifying quantum phases via disentanglement based on deep reinforcement learning}},
  year = {2021},
  eprint = {2107.03542},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.03542},
}
Figure