Abstract
Building large-scale quantum computers requires connecting qubits into complex "resource states," a process hindered by hardware inhomogeneity and the probabilistic nature of quantum entanglement. We present a Reinforcement Learning (RL) framework utilizing a Transformer-based architecture, "Transformer-on-QuPairs," to manage these dynamic quantum resources in real-time. By processing relationships between qubit pairs and accounting for decoherence, the RL agent achieves a 3x performance improvement over traditional scheduling methods. Our results demonstrate that intelligent scheduling is essential for scaling quantum systems, with performance gains increasing as the number of qubits grows.
Introduction
The construction of large-scale quantum computers relies on entangling qubits into specific, high-fidelity patterns known as "cluster states." However, real-world hardware faces two significant challenges: inhomogeneity and probabilistic outcomes. Inhomogeneity arises from fabrication limits, where qubits vary in coherence time and connectivity. Probabilistic outcomes occur because entangling protocols, such as heralded remote entanglement, are stochastic; failures require retries while existing entanglement decays due to decoherence.
Deciding which qubits to pair dynamically is an NP-hard scheduling problem. Static rules fail to adapt to the "ticking clock" of decoherence, where a delay in finding a perfect link can lead to the loss of the entire quantum state. Traditional optimization methods like Static Minimum Spanning Trees (MST) struggle in these environments because they cannot account for the "luck of the draw" in entanglement successes.
Methods
The proposed solution treats scheduling as a game played in a digitized environment, modeled via Monte Carlo Simulations (MCS). The core of the system is the "Transformer-on-QuPairs" architecture, which leverages the self-attention mechanism to identify optimal entanglement paths in a non-uniform grid.
The model incorporates several key components:
- Input Encoding: The agent receives "pre-information" matrices $M_F$ and $M_R$ representing hardware fidelity and success rates, alongside a dynamic state matrix $M_S$ representing the current entanglement network.
- Self-Attention: The Transformer uses multi-head attention to weigh the importance of different qubit pairs, predicting the "cost" or expected error $\epsilon(t)$ of attempting a specific entanglement link.
- Reward Function: The agent is trained to maximize the Quantum Volume ($V_Q$), a metric derived from the maximum cluster size $N_{max}(t)$ and the accumulated system error.
- Dynamic Guidance: Unlike static maps, this RL agent provides real-time, next-step guidance, choosing the path that balances the quality of a link with the speed of construction to beat decoherence.
Results and Discussion
The RL agent was benchmarked against random scheduling, Static MST, and greedy heuristics using high-fidelity simulated datasets grounded in real-world physics (e.g., spin-photon interfaces).
- Performance Gain: The Transformer-on-QuPairs scheduler improved system performance by over 300% compared to rule-based greedy agents.
- Adaptability: In highly inhomogeneous environments, the RL approach significantly outperformed Static MST. While MST works well for perfect hardware, it lacks the flexibility to handle the stochastic failures inherent in real quantum systems.
- Scalability and Transfer Learning: As the system size increased from 40 to 160 qubits, the performance gap between the RL agent and the random baseline grew from 3.20x to 4.56x. Furthermore, agents trained on smaller systems could be effectively "upscaled" to larger systems, suggesting that the learned scheduling strategies are generalizable.
Conclusion
This work establishes RL-driven scheduling as a critical component for the next generation of scalable quantum processors. While the $N^2$ scaling of qubit pairs may present computational bottlenecks for massive systems, future integration with Monte Carlo Tree Search (MCTS) could provide even better look-ahead capabilities. Intelligent, real-time management of "messy" qubits is a promising path toward practical, large-scale quantum computation.
Summary updated