Abstract
Optimizing the Hamiltonian schedule in Adiabatic Quantum Computing (AQC) is critical for solving complex combinatorial problems like prime factorization. This paper presents a reinforcement learning (RL) framework using the Soft Actor-Critic (SAC) algorithm to automatically configure these schedules. By focusing on "hard" factorization instances and employing a reward signal based on worst-case success probability, the proposed method doubles the success rates compared to standard linear schedules. Furthermore, the framework demonstrates strong transferability, allowing models trained on small-scale systems to accelerate optimization in larger qubit systems.
Introduction
Prime factorization remains a cornerstone of modern cryptography, specifically the RSA algorithm. While Shor’s algorithm provides a theoretical polynomial-time solution, its requirement for fault-tolerant quantum hardware makes it impractical for the current Noisy Intermediate-Scale Quantum (NISQ) era. Adiabatic Quantum Computing (AQC) offers a more immediate alternative by encoding factorization into the ground state of a spin Hamiltonian.
However, this encoding often results in a "spin glass" landscape— a rugged energy surface where standard linear or quadratic evolution schedules frequently trap the system in metastable states. This paper addresses the challenge of navigating these landscapes by using reinforcement learning to dynamically design evolution schedules that can "rescue" hard factorization instances from these local minima.
Methodology
The authors frame the AQC schedule design as a Markov Decision Process (MDP). The process is divided into two distinct phases: instance classification and schedule optimization.
Two-Phase Framework
- Phase 1: Instance Selection: Factorization problems (composite numbers $N$) are mapped to Ising Hamiltonians. Using classical Simulated Annealing and standard AQC, instances are categorized into "easy" and "hard" groups based on their baseline failure rates.
- Phase 2: RL Configuration: A Soft Actor-Critic (SAC) agent is employed to tune the parameters of the Hamiltonian evolution schedule. The agent receives a reward based on the minimum log-success probability across the identified hard instances, forcing it to optimize for the worst-case scenario rather than just the average.
Datasets and Encoding
The study utilizes synthetic prime factorization instances for composite numbers ranging from $N = 49$ to $N = 633$, requiring between 5 and 11 qubits. These are encoded into Quadratic Unconstrained Binary Optimization (QUBO) problems and subsequently translated into Ising Hamiltonians.
Key Insights and Results
Non-Linear Schedule Optimization
The RL agent consistently converges on highly non-linear schedules. Unlike the standard linear approach, the learned schedule features a significant plateau at the beginning of the evolution (approximating a 7th-order power law). This slow initial phase is hypothesized to allow the quantum system more time to tunnel through the energy barriers characteristic of the spin glass landscape.
Performance Gains
The RL-configured schedules achieved a doubling of success probabilities for notoriously difficult 7-qubit instances (e.g., $N=55, 77, 91$). The SAC algorithm demonstrated remarkable efficiency, reaching convergence in approximately 1,600 measurement steps—orders of magnitude faster than many previous RL-based quantum control methods.
Transferability and Scalability
A major finding is the high degree of transferability of the learned physics. Neural network weights (actor and critic) trained on a 5-qubit system could be used to initialize agents for 7-qubit or 9-qubit systems. This transfer learning approach allowed the agent to reach near-optimal performance in as few as 150 steps, suggesting that the underlying features of the factorization energy landscape are consistent across scales.
Discussion and Future Work
While these results are promising, they are currently based on classical simulations of quantum systems. Real-world deployment on hardware like D-Wave systems will need to account for environmental noise and decoherence, which were not modeled in this study. Future work will focus on testing the robustness of these RL-configured schedules on physical quantum annealers to determine if the automated design can mitigate hardware-specific noise while scaling to larger, cryptographically relevant integers.
Summary updated