Using Reinforcement Learning to Guide Graph State Generation for Photonic Quantum Computers

Yingheng Li, Yue Dai, Aditya Pawar, Renjie Dong‬, Jun Yang, Youtao Zhang, Xulong TangarXiv 2024 · DOI · arXiv

Abstract

Photonic quantum computer (PQC) is an emerging and promising quantum computing paradigm that has gained momentum in recent years. In PQC, which leverages the measurement-based quantum computing (MBQC) model, computations are executed by performing measurements on photons in graph states (i.e., sets of entangled photons) that are generated before measurements. The graph state in PQC is generated deterministically by quantum emitters. The generation process is achieved by applying a sequence of quantum gates to quantum emitters. In this process, i) the time required to complete the process, ii) the number of quantum emitters used, and iii) the number of CZ gates performed between emitters greatly affect the fidelity of the generated graph state. However, prior work for determining the generation sequence only focuses on optimizing the number of quantum emitters. Moreover, identifying the optimal generation sequence has vast search space. To this end, we propose RLGS, a novel compilation framework to identify optimal generation sequences that optimize the three metrics. Experimental results show that RLGS achieves an average reduction in generation time of 31.1%, 49.6%, and 57.5% for small, medium, and large graph states compared to the baseline.

The Challenge of Photonic Graph State Generation

Photonic Quantum Computers (PQCs) operate under the Measurement-Based Quantum Computing (MBQC) model. Unlike traditional gate-based models that start with unentangled qubits, MBQC begins with a large, pre-entangled "graph state" of photons. Logic is then executed by performing specific measurements on these photons.

Generating these complex graph states deterministically requires quantum emitters, such as quantum dots. However, determining the optimal sequence of operations to construct a specific graph state is a significant challenge. The search space for an optimal sequence grows factorially ($V!$) with the number of photons ($V$). Furthermore, there are critical trade-offs in fidelity. Previous optimization tools, such as the "Stabilizer Solver," focused almost exclusively on minimizing the number of emitters. They largely ignored two other vital metrics: generation time, which leads to decoherence, and the number of CZ gates, which introduces gate errors.

Comparison of generation sequences based on different emission orders.

As shown in the figure above, different emission orders for the same target graph can lead to vastly different hardware requirements, including the number of emitters and CZ gates. As graph states scale, existing solvers either become computationally infeasible or produce inefficient sequences that compromise the accuracy of the quantum computer.

RLGS: A Reinforcement Learning Framework

To address these challenges, researchers developed RLGS (Reinforcement Learning-guided Graph State generation). RLGS treats the generation of graph states as a sequential decision-making problem, utilizing a Deep Q-Network (DQN) to navigate the complex search space.

Core Components of RLGS

  1. High-Level Graph Operations: Instead of dealing with raw quantum gates directly, RLGS utilizes six high-level graph operations, such as Emitter Swap and various types of absorption/emission. These operations allow the framework to "deconstruct" a target graph state into a simple starting point. This deconstruction sequence is then reversed to form the actual generation plan.
  2. GNN State Encoding: The framework employs a Graph Isomorphism Network (GIN) to encode the current state of the photonic graph into a vector. This allows the RL agent to effectively "understand" the structural relationships and entanglement patterns between photons.
  3. Multi-Objective Reward Function: The RL agent is trained using a reward function that penalizes high generation time, excessive emitter counts, and high CZ gate counts, encouraging the discovery of balanced, high-fidelity sequences.
  4. Receptive Field Strategy: To maintain scalability for massive graphs, RLGS uses a "receptive field" approach. The agent only considers operations on a subset of photons in the immediate vicinity of the active emitter, keeping the action space constant regardless of the total number of photons.
Examples of graph operations and corresponding quantum gates.

Performance and Scalability

RLGS was evaluated against six representative quantum applications, including the Quantum Fourier Transform (QFT) and Quantum Approximate Optimization Algorithm (QAOA), across small (<50 photons), medium (>100 photons), and large (>200 photons) scales.

Key Results

  • Significant Efficiency Gains: RLGS achieved average reductions in generation time of 31.1% for small graphs, 49.6% for medium graphs, and 57.5% for large graphs compared to the baseline.
  • Reduced Hardware Overhead: Despite the baseline being optimized specifically for emitter count, RLGS reduced the number of emitters by 13.9% to 17.5%. It also achieved a 37.7% to 57.8% reduction in CZ gate counts.
  • Generalization and Parallelism: A key insight is that an RL agent trained on small benchmarks (5-6 qubits) can generalize effectively to much larger benchmarks (50+ qubits). Furthermore, RLGS naturally discovers sequences where multiple emitters can operate in parallel, which is the primary driver for the reduced generation time.
  • Scalability: In tests with synthetic graphs of up to 1200 photons, the baseline Stabilizer Solver failed to find valid sequences, whereas RLGS successfully identified optimal plans.

Conclusion and Future Directions

RLGS represents a major advancement in the compilation of graph states for photonic quantum hardware. By optimizing for multiple fidelity-impacting metrics simultaneously and leveraging the power of RL and GNNs, it provides a scalable path toward utility-scale quantum computing.

Future work will focus on incorporating specific hardware constraints, such as limited emitter connectivity on a chip. Additionally, integrating detailed photon loss models and real-time hardware noise data into the reward function could further improve the generation sequences for specific physical implementations.

Summary updated

Abstract

Photonic quantum computer (PQC) is an emerging and promising quantum computing paradigm that has gained momentum in recent years. In PQC, which leverages the measurement-based quantum computing (MBQC) model, computations are executed by performing measurements on photons in graph states (i.e., sets of entangled photons) that are generated before measurements. The graph state in PQC is generated deterministically by quantum emitters. The generation process is achieved by applying a sequence of quantum gates to quantum emitters. In this process, i) the time required to complete the process, ii) the number of quantum emitters used, and iii) the number of CZ gates performed between emitters greatly affect the fidelity of the generated graph state. However, prior work for determining the generation sequence only focuses on optimizing the number of quantum emitters. Moreover, identifying the optimal generation sequence has vast search space. To this end, we propose RLGS, a novel compilation framework to identify optimal generation sequences that optimize the three metrics. Experimental results show that RLGS achieves an average reduction in generation time of 31.1%, 49.6%, and 57.5% for small, medium, and large graph states compared to the baseline.
BibTeX
@misc{li2024using,
  author = {Yingheng Li and Yue Dai and Aditya Pawar and Renjie Dong‬ and Jun Yang and Youtao Zhang and Xulong Tang},
  title = {{Using Reinforcement Learning to Guide Graph State Generation for Photonic Quantum Computers}},
  year = {2024},
  eprint = {2412.01038},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.01038},
}
Figure