Sequential Control of Quantum States via Bellman Recursion

13 concepts5 papers1 proposal

In classical control theory, guiding a system to a target state typically involves applying continuous external forces. In quantum mechanics, however, we have a unique alternative lever: measurement back-action. When we perform a measurement on a quantum system, its state changes stochastically. By choosing which measurement to perform based on the current state, we can actively steer the system toward a desired target.

This adaptive process transforms quantum state manipulation into a sequential decision-making problem. By framing this challenge through the lens of Markov Decision Processes (MDPs), researchers have unlocked the ability to use dynamic programming to find optimal control strategies.

The Quantum Markov Property

Why can we treat a sequence of quantum measurements as a classical Markov Decision Process? The key lies in how a quantum state evolves.

A quantum system's state is fully captured by its density matrix, denoted as $\rho$. When an action (such as a specific measurement) is taken, the system collapses into a new state. According to Born's rule, the probability of transitioning to this new density matrix depends only on the current density matrix $\rho$ and the chosen measurement. It does not depend on the history of how the system arrived at $\rho$.

As demonstrated by Shuangshuang Fu et al. in Feedback policies for measurement-based quantum state manipulation, this memoryless property means that optimal feedback policies are strictly Markovian. Because the current density matrix encapsulates all necessary information for future prediction, the control problem simplifies dramatically: we do not need to track the entire history of the system's evolution to make an optimal choice at the current step.

[Current Density Matrix ρ] ---> [Choose Measurement] ---> [Stochastic Collapse to ρ']
           ^                                                            |
           |_______________________Next Step____________________________|

Solving the Control Problem via Backward Recursion

If we try to find the optimal sequence of measurements by looking forward from an initial state, we quickly run into an exponential explosion of possibilities due to the branching outcomes of every measurement. To make the computation tractable, we must leverage Bellman's principle of optimality and work backward.

Instead of planning forward, we evaluate the system from the final time step $T$ back to the beginning. If we know the value or utility of being in any final quantum state at time $T$, we can step back to time $T-1$ and determine which measurement maximizes the expected value of the resulting state at time $T$.

In Feedback design for quantum state manipulation by measurements, Fu et al. applied Bellman's recursion equations to finite time horizons, mapping the current density matrix to the optimal next measurement choice. This backward recursive approach ensures that we find the globally optimal policy without exhaustively simulating every forward path.

This framework was rigorously formalized for general quantum environments by Shenggang Ying in Optimal Policies for Quantum Markov Decision Processes. Ying extended the principle of optimality to "quantum MDPs" (qMDPs) where states are density matrices, providing a mathematical foundation for decomposing multi-step quantum decision processes into recursive, solvable sub-problems.

Flexible Objectives: Beyond Success Probability

Framing quantum control as an MDP does more than just maximize the raw probability of hitting an exact target state; it allows controllers to optimize for a variety of goals.

In Feedback design for quantum state manipulation by measurements, the authors demonstrated that the Bellman recursion framework can easily accommodate alternative objective functions:

  • Stochastic Shortest Path: Minimizing the expected number of measurement steps required to reach a target state.
  • Fidelity Optimization: Maximizing the average expected fidelity of the final state relative to a target state, which is highly valuable when an exact match is mathematically impossible or too costly.

Generalizing and Scaling Quantum Policies

As the theoretical foundations of qMDPs settle, research has shifted toward making these systems computationally tractable and exploring how quantum hardware itself can accelerate dynamic programming.

To address the challenge of computing policies in high-dimensional quantum systems, Naci Saldı et al. developed a verification theory in Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies. They introduced new, computationally tractable classes of policies—such as open-loop and classical-state-preserving closed-loop policies—offering a path forward for approximation algorithms that can operate within realistic computational limits.

Interestingly, the relationship between dynamic programming and quantum mechanics also operates in reverse. Instead of using classical computers to control quantum states, we can use quantum computers to solve classical sequential decision problems. For instance, João F. Doriguello et al. in Quantum Algorithm for Stochastic Optimal Stopping Problems with Applications in Finance designed quantum circuits to recursively determine optimal stopping times. By mapping the classical dynamic programming steps onto quantum amplitudes, the authors achieved a nearly quadratic speedup over traditional classical Monte Carlo methods.

Key Foundational Work

Feedback policies for measurement-based quantum state manipulation (2014) — contributed 8 concept elements to this direction.
Feedback design for quantum state manipulation by measurements (2015) — contributed 4 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dynamic_programming_for_quantum_control_aeacfa79,
  title = {dynamic programming for quantum control},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-programming-for-quantum-control/rdr_aeacfa79a39d42d78d2e5f3736b2c424},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_dynamic_programming_for_quantum_control_aeacfa79_v1,
  title = {dynamic programming for quantum control},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-programming-for-quantum-control/rdr_aeacfa79a39d42d78d2e5f3736b2c424/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake