Quantum networks deliver entangled qubit pairs between distant users. Operators face a hard tradeoff between how many pairs per second they deliver (throughput) and how close those pairs are to perfect entanglement (fidelity). Raising fidelity requires purification, which sacrifices several noisy pairs to produce one better pair. That costs throughput, and beyond some number of rounds it fails, because the noisy gates add more error than purification removes. The best routing and purification choices therefore depend on physical quantities such as gate error rates, how fast stored qubits decay, and how lossy each optical link is. All of these drift over time, as satellites move, hardware ages, and weather changes. Learned routers already beat classical search on this problem. A deep Q-network router in SPARQ resolves more requests than shortest-path routing and picks a path in a few milliseconds instead of ninety. That router, however, is trained offline, and nobody has asked what happens when it must keep learning while the physics drifts underneath it.
We propose that forgetting in such a router is low-rank: it lives in a handful of output directions. If so, a continual learner should protect only those directions and leave everything else free to adapt. That avoids the stiffness that standard anti-forgetting penalties impose. The quantum setting makes this an unusually clean test, because analytic results tell us which decision should change when the hardware changes, namely how deep to purify and how many repeaters to use. We can therefore check whether the protected directions line up with physically meaningful structure, not just whether accuracy goes up.
Core Claim
Continual learning research has a well-documented failure. Penalties that anchor a network's weights to their old values prevent forgetting but make the network stiff, which the field calls "rigidity" or intransigence. Fixed Design Analysis of Regularization-Based Continual Learning proves this happens for plain weight-anchoring penalties in two-task linear regression. When tasks differ, raising the penalty lowers forgetting but raises the failure-to-learn term quadratically in the penalty strength. Continual World shows the practical consequence in reinforcement learning. Penalties such as Elastic Weight Consolidation and Memory Aware Synapses give near-zero or negative forward transfer, and anchoring the value network (the "critic") is especially harmful because value targets must move. A Q-network router is entirely a value network, so naive regularization should be at its worst here.
A different line of work suggests an escape. Catastrophic Forgetting is Low-Rank uses neural tangent kernel theory (a linearization describing how a network's outputs move when its weights move) to show that forgetting in supervised classifiers concentrates in 1 to 6 output directions, the top eigenmodes of the old task's kernel. Penalizing drift only along those directions works well on shared-head benchmarks. Our claim is that this result transfers to value-based reinforcement learning in drifting quantum networks, and that it resolves the critic-rigidity problem. Protecting a few directions leaves the Q-function free to track moving targets in the rest of its output space. The claim is falsifiable in a specific way. If router forgetting is not low-rank, the protected subspace will need to be large, and the method will degrade into the same rigidity that Elastic Weight Consolidation shows.
Research Program
The first piece is a testbed. No public continual benchmark exists for entanglement routing, so we build one on the US Backbone and Waxman random topologies used by Fidelity-Guaranteed Entanglement Routing. We add the dynamic link transmissivities used in SPARQ and explicit physical noise. Gate and measurement fidelity is centered on the 0.99 regime that Scaling Limits of Quantum Repeater Networks uses to derive its 115 km reach limit. The memory dephasing time follows the small high-turnover buffers that Towards A High-Performance Quantum Data Center Network Architecture finds optimal. A "regime" is a setting of these physical parameters.
Regime sequences are built to cross the purification diminishing-returns threshold described in Scaling Limits and the capacity contraction proved in On the Capacity Region of a Quantum Switch with Entanglement Purification. As a result, the optimal number of purification rounds actually changes between regimes. Regimes recur, as satellite passes and day/night cycles do, so forgetting has a real cost.
The agent observes the network graph, per-link current fidelity estimates, memory occupancy and pending requests. Its actions are a path plus a per-hop purification depth. Its reward is fidelity-qualified throughput $T_F$, the rate of delivered pairs whose end-to-end fidelity exceeds the application threshold $F_{\min}$. This single number captures the tradeoff, because pairs delivered fast but below threshold earn nothing.
The second piece is a diagnostic study, run before any method is proposed. We fine-tune a router through regime sequences, then compute the empirical neural tangent kernel of the old-regime Q-function on a set of anchor states. We measure what fraction of the drop in old-regime Q-values falls in its top-$k$ eigenmodes. We also project those eigenmodes onto interpretable decision contrasts, such as "purify one more round on this hop" versus "do not." This tells us whether low-rank forgetting holds for value functions and whether it is physically aligned.
The third piece is the method. When a regime is consolidated, we store anchor states $X$, the old Q-values $Q_{\text{old}}(X)$, and the top-$k$ eigenvectors $u_1,\dots,u_k$ of the old kernel Gram matrix on $X$. Training then minimizes the usual temporal-difference loss plus
$$ \mathcal{R}(\theta) = \lambda \sum_{j=1}^{k} \left( u_j^\top \left[ Q_\theta(X) - Q_{\text{old}}(X) \right] \right)^2 , $$
where $Q_\theta(X)$ is the stacked vector of current Q-values on the anchors, $\lambda$ is the penalty weight, and $k$ is chosen as the smallest number of modes capturing a fixed fraction of the measured forgetting energy. All other output directions are unpenalized.
Real networks give no task boundaries, which Continual World flags as a limitation of many methods. We therefore trigger consolidation from a change detector on temporal-difference error and on residuals of observed link fidelity. Known-boundary consolidation is reported as an upper reference. To track the capacity-loss failure mode separately from forgetting, we also log the rank of the penultimate features, following Understanding and Preventing Capacity Loss in Reinforcement Learning.
Baselines
The naive reference is a SPARQ-style DQN fine-tuned continuously with a standard replay buffer and no consolidation. This is what a deployed learned router would do today. It also matters because Continual World reports that simple fine-tuning reached forward transfer of 0.20, better than most specialized methods, so it is not a straw man. Beating it means higher time-averaged $T_F$ over a full regime sequence, and smaller $T_F$ drops when a previously seen regime recurs.
The strongest existing learned competitor is the SPARQ DQN itself, retrained or fine-tuned per regime. SPARQ reports that its DQN resolves 39% more teleportation requests than shortest-path routing and gains 15% in entanglement fidelity over a DQN trained on a static snapshot, with roughly 3.94 ms path-finding latency against 90–93 ms for traditional algorithms. We reimplement its architecture in our simulator rather than port those numbers, since its environment differs.
The strongest classical competitor is Q-LEAP, an extended-Dijkstra router with a multiplicative fidelity metric and integrated purification decisions, from Fidelity-Guaranteed Entanglement Routing. It is recomputed on every snapshot with true current parameters. Because it never learns, it cannot forget, and it serves as a near-oracle for routing quality whose cost is latency. That paper reports no wall-clock number on Lacuna, only that Q-LEAP is orders of magnitude faster than Q-PATH, so we measure its latency ourselves on identical hardware. Within its latency budget, under explicit memory dephasing, the learned method must approach or exceed Q-LEAP's $T_F$.
The continual learning baselines are three. Elastic Weight Consolidation on the Q-network is the rigidity-prone reference. InFeR is the plasticity-preserving reference. A large replay buffer retaining all past regimes is the memory-based reference. The key ablations replace the top-$k$ spectral penalty with a full function-space penalty on all anchor outputs (same anchors, $k$ = all) and with a random $k$-dimensional subspace. These two isolate whether the specific low-rank directions matter. Every learned method shares the simulator, network architecture, number of environment steps, and a matched hyperparameter search of equal trial count over $\lambda$ or its equivalent, all tuned on separate held-out regime sequences.
Why not just…?
Why not just recompute Q-LEAP on every network snapshot? A classical router with fresh parameters has no forgetting problem at all, and this is the most serious alternative. Its weakness is time. SPARQ measured traditional path search at 90–93 ms against 3.94 ms for a learned router. The data center study shows that stored pairs decay fast enough that small, fast-turnover buffers win, so slow decisions cost fidelity. Our simulator models this decay explicitly, so the comparison directly measures whether latency translates into lost $T_F$. If Q-LEAP still wins once its compute time is charged against memory decay, that is a real negative result for learned routing, and we will report it as such.
Why not just feed the measured hardware parameters to the policy? If gate fidelity and dephasing time were observed exactly, a parameter-conditioned policy trained over all regimes would need no continual learning. We test this directly with an oracle-conditioned DQN and with one conditioned on noisy, delayed estimates, modeling the fact that gate and memory characterization is slow and indirect. If the noisy-conditioned policy matches our method, the proposal's contribution shrinks to cases where conditioning variables are missing. We consider that a meaningful boundary rather than a failure.
Why not just keep a big replay buffer or periodically retrain? Replay over all regimes is a strong rehearsal method when regimes recur. Retraining from scratch discards forward transfer: Continual World reports a reference transfer of 0.46 that perfect feature reuse should reach, while most regularization methods scored near zero. Replay is included as a baseline at matched memory. Recovery time after each regime shift measures the cost of retraining, since throughput is near zero while a fresh router converges.
Why not just use Elastic Weight Consolidation? Theory and evidence both say it will stiffen a value network. The fixed-design analysis shows its quadratic intransigence ceiling, Continual World shows it chokes critics, and Catastrophic Forgetting is Low-Rank reports around 78% shared-head Split-MNIST accuracy for spectral regularization against about 20% for EWC and Synaptic Intelligence. Whether that gap survives in temporal-difference learning is exactly what we test.
Proposed Experiments
The first decisive check is diagnostic. On recurring regime sequences across both topologies, we test whether the top 1–6 eigenmodes of the old-regime kernel capture most of the drop in Q-values. The supervised result reported 50–90% of forgetting energy in such modes. If router forgetting needs dozens of modes, the core claim fails and we report that.
The second check is the physical alignment test. We test whether the dominant modes project mainly onto purification-depth contrasts on hops whose gate fidelity crossed the diminishing-returns point. If they do, the method has learned physics-shaped structure; if not, low-rank forgetting would be a generic optimization artifact.
The third check is the main comparison. We measure time-averaged $T_F$, average delivered fidelity, the $T_F$ drop on regime recurrence (forgetting), steps to recover 90% of a regime's eventual $T_F$ (plasticity), and penultimate feature rank. Each is reported over at least five seeds with confidence intervals, under both detected and known boundaries. The proposal succeeds if top-$k$ spectral protection matches replay and Elastic Weight Consolidation on forgetting while matching fine-tuning and InFeR on recovery speed, beats the full-penalty and random-subspace ablations, and stays within Q-LEAP's $T_F$ once Q-LEAP's latency is charged against memory decay.
References
- SPARQ: Efficient Entanglement Distribution and Routing in Space–Air–Ground Quantum Networks
- Fixed Design Analysis of Regularization-Based Continual Learning
- Continual World: A Robotic Benchmark For Continual Reinforcement Learning
- Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation
- Fidelity-Guaranteed Entanglement Routing in Quantum Networks
- Scaling Limits of Quantum Repeater Networks
- Towards a High-Performance Quantum Data Center Network Architecture
- On the Capacity Region of a Quantum Switch with Entanglement Purification
- Understanding and Preventing Capacity Loss in Reinforcement Learning