Stochastic optimization for learning quantum state feedback control

Ethan N. Evans, Ziyi Wang, Adam G. Frim, Michael R. DeWeese, Evangelos A. TheodorouarXiv 2021 · DOI · arXiv

Abstract

High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.

Introduction

Preparing and stabilizing specific quantum states, such as entangled Bell states, is a foundational goal in quantum computing. However, quantum systems are notoriously fragile, susceptible to environmental noise and decoherence. While traditional "open-loop" control sequences are effective under ideal conditions, real-world applications require robust state feedback control—the ability to adjust control signals in real-time based on the system's evolving state.

Implementing such feedback is technically challenging due to measurement back-action, where observing a system can collapse its state. This work utilizes Weak Continuous Quantum Non-Demolition (QND) measurements to extract information without total collapse, though this introduces complex stochastic noise (Wiener diffusion). Furthermore, standard control algorithms often fail in quantum contexts because they require inverting noise covariance matrices that are frequently singular.

Methodology: Quantum Gradient-based Adaptive Stochastic Search (QGASS)

To overcome these hurdles, we introduce Quantum Gradient-based Adaptive Stochastic Search (QGASS). This framework treats quantum control as a reinforcement learning-style optimization task. Instead of optimizing neural network parameters directly via standard backpropagation—which can be unstable given the non-differentiable nature of some quantum cost functions—QGASS optimizes the distribution of the controller's weights.

Schematic of the QGASS framework, illustrating the iterative process of sampling neural network parameters, simulating closed-loop dynamics, and updating the sampling distribution based on trajectory performance.

As shown in Figure 1, the process involves maintaining a Gaussian distribution over the neural network's weights. Multiple candidate controllers are sampled and run in parallel simulations. By applying a "log-trick" from stochastic optimization, the mean of the parameter distribution is updated toward configurations that yield superior performance. This approach bypasses the mathematical "dead ends" of matrix inversion and is highly parallelizable, making it suitable for modern GPU acceleration.

Experimental Setup and Benchmarking

The framework was evaluated using a simulated two-qubit system (implemented via the QuTiP library) with the goal of reaching and stabilizing the symmetric "up-down, down-up" Bell state. We compared QGASS against a landmark analytical feedback control law established by Mirrahimi and Van Handel (2007).

Key Results and Discussion

QGASS demonstrated significant performance gains over the analytical benchmark across several metrics:

  • Efficiency: The learned policy achieved stabilization roughly 10x faster than the benchmark.
  • Control Effort: Stabilization was reached using approximately 1/12th of the cumulative control energy, as the network discovered an emergent "bang-bang" impulsive control strategy rather than constant pushing.
  • Precision: The final states reached by QGASS exhibited lower statistical variance, indicating more reliable and repeatable control.
Comparison of running state and control costs over time. QGASS (red) achieves lower state costs faster and with significantly less control effort than the Mirrahimi benchmark (blue).

As illustrated in Figure 4, the state cost for QGASS levels off much earlier and at a lower value than the benchmark, while the control cost remains nearly flat after the initial pulse. Furthermore, the trained network proved robust; although trained on only 1,000 timesteps, it remained stable for 100,000 timesteps during testing, suggesting it learned a fundamental physical control law rather than a simple memorized sequence.

Conclusion and Future Work

QGASS provides a robust, parallelizable framework for real-time quantum feedback control that outperforms traditional analytical methods. While currently demonstrated on two-qubit systems, future work will focus on scaling the approach to many-qubit architectures. Addressing the exponential growth of the Stochastic Master Equation simulation and implementing these neural networks on low-latency hardware like FPGAs are critical steps toward practical laboratory deployment.

Summary updated

Abstract

High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
BibTeX
@misc{evans2021stochastic,
  author = {Ethan N. Evans and Ziyi Wang and Adam G. Frim and Michael R. DeWeese and Evangelos A. Theodorou},
  title = {{Stochastic optimization for learning quantum state feedback control}},
  year = {2021},
  eprint = {2111.09896},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.09896},
}
Figure