A quantum system can be steered by measurement alone. Each measurement randomly "kicks" the state, and a controller that picks the next measurement based on what it just saw can drive a qubit to a target state with high probability. The standard tool for designing such controllers is dynamic programming, which here means working backward from the final step to compute, for every possible quantum state, the best measurement to make next. Every existing formulation optimizes one number at a time: the probability of success, the expected fidelity (how close the final state is to the target), or the expected number of steps. On real hardware these goals conflict, because each extra step exposes the state to noise. The natural response is to build new multi-objective dynamic programming that keeps whole sets of Pareto-optimal values at each state.
We argue that this machinery is mostly unnecessary, and we propose to show it with an efficient algorithm. For objectives that are expectations, such as success probability and expected step count, the set of outcomes achievable by policies that are allowed to randomize is convex. A short sequence of ordinary single-objective Bellman solves therefore recovers the entire trade-off front exactly. In each solve, time is charged a price $\lambda$, and the prices are chosen at the front's corners rather than on a grid. The genuinely multi-objective part is risk: tail probabilities such as "finishes too late," which a price on expected time cannot see. For that part we propose augmenting the state with one scalar, which again reduces the problem to single-objective dynamic programming. The deliverables are a procedure that computes the exact front for measurement-based qubit and few-qubit control under decoherence, with a provable bound on the number of Bellman solves, and a clear map of where scalar pricing stops working.
Core Claim
We consider measurement-based feedback on a quantum Markov decision process: a sequential decision problem whose state is the density matrix $\rho$ and whose actions are measurements. Between measurements, a decoherence channel (dephasing or amplitude damping) acts on the state. There is a "stop" action that ends the episode and scores the final fidelity. For a price $\lambda \ge 0$ per measurement step, the priced Bellman recursion is
$$V^{\lambda}_t(\rho) = \min\Big\{\, 1-F(\rho),\;\; \lambda + \min_{a}\sum_{o} p(o\mid\rho,a)\, V^{\lambda}_{t+1}(\rho_{a,o}) \Big\},$$
with the following notation. $F(\rho)$ is the fidelity to the target. $a$ ranges over the available measurements. $o$ is the outcome, and $p(o\mid\rho,a)$ is its probability under Born's rule. $\rho_{a,o}$ is the post-measurement state after the noise channel. The horizon ends at step $N$ with $V^{\lambda}_N = 1-F$.
The optimal priced cost $g(\lambda) = \min_\pi \big(\mathbb{E}_\pi[1-F] + \lambda\,\mathbb{E}_\pi[\tau]\big)$, where $\tau$ is the number of steps taken, is a lower envelope of lines, one per policy. It is therefore concave and piecewise linear in $\lambda$. Its breakpoints correspond one-to-one with the corner policies of the (infidelity, expected time) front. Randomizing between adjacent corner policies fills in the segments between them, and because the achievable set of expected outcomes under randomized policies is convex, no front point is missed.
A breakpoint search finds all $K$ corners with $2K-1$ Bellman solves. The search solves at the two extreme prices, sets $\lambda$ equal to the slope of the chord between the two resulting outcome points, and solves again, recursing until no new corner appears. This is the classical "dichotomic" scheme from bi-objective optimization. What is new here is applying it to quantum Bellman recursions and measuring $K$ on physical problems.
This claim contradicts part of the motivation assembled around this intersection. It is often argued, drawing on Distributional Pareto-Optimal Multi-Objective Reinforcement Learning, that weighted sums miss non-convex parts of the front. That argument holds for deterministic policies and for criteria that are not expectations. It does not hold for expected fidelity and expected time under randomized policies. We therefore use that paper as a constraint that marks where our claim ends, which is at risk and distributional criteria. We do not use it as motivation for a vector-valued Bellman operator.
Research Program
The method comes from three sources. Feedback policies for measurement-based quantum state manipulation proves that optimal measurement policies depend only on the current density matrix, so a Bellman recursion over $\rho$ is sufficient. Feedback design for quantum state manipulation by measurements solves the success-probability and minimal-arrival-time problems separately for a qubit. We reuse its benchmark, and its two separate optima become the two endpoints of our front. Optimal Policies for Quantum Markov Decision Processes gives the general formulation and a matrix-product formula (its Theorem 3) that computes outcome probabilities in an $n^2 \times n^2$ space. We use that formula for the few-qubit experiments. Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies proves that finite-action approximations converge to the continuous problem. That result justifies discretizing the continuous measurement directions, and we will report how the front moves as the discretization is refined. None of these four papers considers more than one objective at a time, and none includes decoherence between measurements. Our work adds both.
The first thrust is exact computation for a qubit. Dephasing or amplitude damping toward a target on the Bloch sphere's axis is symmetric about that axis. The value function therefore lives on a two-dimensional disk (radius and polar angle) rather than on the full Bloch ball. We solve it on a fine grid with interpolation, and measurements can be projective or weak with a tunable strength. In the noiseless projective case the reachable states form a finite tree, so the recursion is exact.
The second thrust scales to two and three qubits. We use finite horizons, the exact finite-tree recursion of Ying, and pruning of states whose probability of being reached is negligible. The goal is to test whether the number of corners $K$ stays small as the system grows. If it does, the cost of the full front is a small multiple of one single-objective solve.
The third thrust covers risk. We want the conditional value-at-risk of arrival time, meaning the expected arrival time over the worst $\alpha$ fraction of runs. We compute it with the standard auxiliary-threshold trick: an outer one-dimensional minimization over a threshold $\eta$ wraps an inner single-objective Bellman solve. In that inner solve the state carries the remaining budget and the terminal penalty is $\max(\tau-\eta,0)$.
Baselines
The naive reference point is the open-loop measurement sequence from Fu et al.: a fixed schedule of measurement bases with no feedback. On Benchmark Q1 it reaches a success probability of about 0.56 at $N=3$ and about 0.80 at $N=10$ (Feedback design for quantum state manipulation by measurements). We re-run it under each noise model to anchor the bottom of every plot.
The strongest existing method is Fu et al.'s single-objective optimal feedback. It reports a success probability of 0.9968 at $N=10$ (about 0.66 at $N=3$) and a minimal average arrival time of 3.8 steps. These are two isolated points. Beating it cannot mean exceeding 0.9968, since that value is optimal for its criterion. Instead, our front must reproduce both points as its endpoints to three decimals in the noiseless setting. It must also expose the intermediate policies, for example the fewest expected steps needed to guarantee a success probability of at least 0.95, which no single solve reports. For the Quantum Robot Grid World (Benchmark Q2) there is no reported result on Lacuna, so we will reproduce Ying's scalar-reward optimum and treat it as one point on the front.
The exact reference is set-valued Pareto dynamic programming. It stores the full set of non-dominated (infidelity, time) vectors at every state and merges these sets through the recursion. It is exact but its memory grows with set size. We run it on the same grids, horizons, and measurement pools. If the breakpoint search plus mixing matches its front within grid tolerance, measured by an additive epsilon indicator below $10^{-4}$, while using a fraction of the time, the claim holds.
We also run two black-box competitors on identical simulators and an equal wall-clock budget. The first is NSGA-II over neural or lookup-table policies. The second is a surrogate-assisted evolutionary search in the style of Un-evaluated Solutions May Be Valuable in Expensive Optimization. Because that method ranks candidates by a scalar surrogate, we run it once per price, which is the fair adaptation. Our method beats these if it reaches higher hypervolume at every budget.
Two ablations isolate the key ingredients. The first restricts the method to deterministic policies (no mixing) and measures how much of the front is lost. The second replaces breakpoint search with a uniform grid of 100 prices and measures solves spent against front error.
Why not just…?
Why not just sweep a grid of weights? For a single qubit this is essentially our method with a worse choice of prices, and a 100-point sweep is cheap. It is a real competitor there, and we will say so. The difference appears when each Bellman solve is expensive, as at two to three qubits where Ying's complexity bound $O(|O|^{N+1}|A|^N n^6)$ applies. There, the breakpoint search spends exactly $2K-1$ solves and certifies that the front is complete, while a fixed grid either wastes solves on flat stretches or misses narrow corners. The uniform-grid ablation settles which regime we are in.
Why not just run NSGA-II on the simulator? Black-box evolutionary search ignores the fact that the optimal policy depends only on the current state. It therefore has to rediscover the backward structure from whole-trajectory returns. Hao et al. document how sample-hungry such searches are when evaluations are costly, and our matched-budget comparison measures the resulting hypervolume gap directly.
Why not just reduce to a deterministic problem as in eigenstate-reduction model predictive control? Tractable Infinite-Horizon Stochastic Model Predictive Control for Quantum Filtering via Eigenstate Reduction collapses continuous-measurement control to a single-step fidelity problem solved by Pontryagin's principle, and it reports a fidelity of 0.9749 on an 11-level system. That reduction relies on the dynamics and measurements sharing eigenstates and on smooth control fields. Our actions are discrete choices of measurement basis, and the noise does not commute with those bases. The reduction also yields one scalar optimum, not a front.
Why not just build a vector-valued Bellman operator? For expected-value criteria it computes the same front, only more expensively. Our comparison against it is the central experiment. It remains necessary only for criteria that depend on the distribution of outcomes, and even there we test whether the one-scalar augmentation matches it.
Proposed Experiments
The first check reproduces Benchmark Q1 noiselessly. Pass means the front's endpoints equal 0.9968 at $N=10$ and 3.8 expected steps. We report $K$ and the solve count, together with front agreement against set-valued dynamic programming.
We then add dephasing and amplitude damping at several rates per measurement step, together with weak measurements, and track how the front bends. The expected result is that high-fidelity endpoints become unreachable, and that the price at which stopping early becomes optimal shifts in a way we can predict from the noise rate.
On the Grid World and on two- and three-qubit state preparation, we measure $K$ as the system grows, along with wall-clock cost relative to one single-objective solve and to the black-box baselines at equal budget.
The decisive risk experiment optimizes the conditional value-at-risk of arrival time at $\alpha = 0.1$. We compare the best policy from the expected-value front, the policy from the augmented single-objective solve, and a distributional set-valued baseline in the spirit of DPMORL. The outcome decides the proposal's boundary. If the augmented solve matches the distributional baseline, then single-objective dynamic programming covers every trade-off we examined. If a gap persists, that gap is the concrete case where quantum feedback control genuinely needs multi-objective machinery, and we will report it as such.
References
- Distributional Pareto-Optimal Multi-Objective Reinforcement Learning
- Feedback policies for measurement-based quantum state manipulation
- Feedback design for quantum state manipulation by measurements
- Optimal Policies for Quantum Markov Decision Processes
- Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies
- Un-evaluated solutions may be valuable in expensive optimization
- Tractable Infinite-Horizon Stochastic Model Predictive Control for Quantum Filtering via Eigenstate Reduction