A mixture of experts is a network in which a small "router" decides which of several sub-networks (experts) handles each input. Its best-known failure is expert collapse: the router sends nearly everything to one or two experts, or the experts end up computing the same function, so the extra capacity is wasted. Quantum versions of this architecture now exist. In QMoE, both the router and the experts are small parameterized quantum circuits, and the router holds a superposition over expert choices instead of picking one. QMoE reports accuracy only. Nobody has checked whether these quantum routers collapse. There is also a more basic unanswered question: does the superposition do any work that a classical coin flip over experts could not?
This note proposes answering both questions with one number, quantum discord between the router qubits and the readout qubit. Discord is the part of the correlation between two quantum systems that is necessarily destroyed when one of them is measured. If the router register is measured before the experts run, the model becomes a classical stochastic mixture of experts. The mutual information between router and output that this measurement destroys is at least the discord. So discord is a floor on what the quantum router contributes beyond classical sampling. Both classical collapse modes drive discord to zero, because starvation leaves no router uncertainty and identical experts leave no router–output correlation. A third, quantum-specific mode drives discord to zero while accuracy and load balance look healthy: the router behaves like a classical random choice. Two-qubit discord is normally an optimization over measurements. For Bell diagonal states (two-qubit states diagonal in the Bell basis), however, Gårding et al. show that it equals the relative-entropy distance to the nearest "classical-router" state, and it has a closed form. That closed form gives a differentiable diagnostic and regularizer whose meaning is exactly "how far is this model from a classically routed one."
Core Claim
We claim three things. The first is that QMoE-style routers show collapse modes, namely amplitude concentration on one expert index and convergence of expert unitaries, that neither QMoE's accuracy metric nor classical load statistics computed on measured router outcomes can tell apart from healthy training. The second is that the router–readout discord, computed through a Bell-diagonal projection, predicts across training runs how much accuracy is lost when the router is replaced by mid-circuit measurement, which amounts to classical sampling. The third is that adding a discord floor as a regularizer improves the quantum model over porting classical balancing losses. The first two claims are diagnostic and are the core contribution. The third is conditional: we test it, but it may fail, since discord and accuracy need not align.
Research Program
The object of study is the two-qubit reduced state $\rho_{RO}$ of one router qubit $R$ and the readout qubit $O$ after the expert block. With 2 or 4 experts there are one or two router qubits, so every router qubit yields one such state. We write the state's correlation matrix as $T_{ij} = \mathrm{tr}(\rho_{RO}\,\sigma_i\otimes\sigma_j)$, where the $\sigma_i$ are Pauli matrices. Local unitaries change neither discord nor collapse, and they can make $T$ diagonal. We then form the Bell-diagonal projection by dropping the local Bloch vectors and keeping the diagonal correlations $c_1, c_2, c_3$. This projected state has Bell-basis eigenvalues $\lambda_k$, and its discord in bits is
$$D_{\mathrm{BD}} = 2 + \sum_{k=1}^4 \lambda_k \log_2 \lambda_k - \Big[\tfrac{1+c}{2}\log_2(1+c) + \tfrac{1-c}{2}\log_2(1-c)\Big], \qquad c = \max_i |c_i|.$$
The first two terms are the mutual information, since both marginals of a Bell-diagonal state are maximally mixed. The bracket is the best classical correlation obtainable by measuring the router. For Bell-diagonal states this value coincides with the asymmetric relative entropy of discord, which is the equivalence established on IBM hardware by Gårding et al.. The projection itself is an approximation. It is exact only when the local Bloch vectors vanish, which happens precisely when the router is balanced, and it is not guaranteed to bound the true discord. We therefore always report it next to exact numerical discord and treat the gap between them as a measured quantity, not an assumption.
We would build a QMoE reimplementation following the four-stage design in QMoE: encoding, routing circuit, controlled experts, and aggregation. It would be instrumented with four quantities. The first is exact discord. The second is $D_{\mathrm{BD}}$. The third is the router-measurement information loss $I(\rho_{RO}) - J_z$, where $J_z$ is the classical correlation that survives measuring the router in the computational basis. By construction this loss is at least the exact discord, which is the operational anchor of the whole proposal. The fourth set is classical collapse statistics on measured router outcomes: routing entropy and a Gini coefficient over experts. We add quantum analogues of the classical homogenization measures, namely pairwise fidelity between expert unitaries as a counterpart to the output orthogonality of Guo et al. and the embedding similarity of Agarap & Azcarraga. The regularizer adds $\beta\max(0, \delta - D_{\mathrm{BD}})$ to the task loss, where $\beta$ is a weight and $\delta$ is a target floor, so that training keeps the router non-classical without forcing any particular load pattern.
The two directions play different roles. The Bell-diagonal discord equivalence supplies the method: a closed-form, interpretable measure of distance to a classical router. Expert collapse supplies the problem and the classical baselines. The robustness of discord under decoherence, where it can persist or even grow while entanglement dies (Zhou; Jiménez et al.), is the reason to prefer discord over entanglement measures on noisy mixed router states. The heavier constructions, minimal discord over cross-symmetric extensions (Xiong et al.) and unextendible entanglement (unextendibility), constrain the design rather than supply it. They target entanglement, not the router-classicality question, and the extension optimization is intractable beyond small dimensions. We therefore deliberately do not use them.
Baselines
The do-nothing reference is QMoE trained with the task loss alone, reproduced on the same 8×8 splits. QMoE reports 98.62% on MNIST-2, 62.92% on MNIST-4, and 63.81% on Fashion-4, against monolithic-network baselines of 94.89%, 51.47%, and 53.65% (QMoE). Our reproduction must land near these numbers before anything else is interpreted.
The strongest existing collapse method we can port is φ-balancing (Chen et al.), a population-level negative-entropy potential on expert usage tracked by an exponential moving average. We apply it to the router's computational-basis probabilities. It is the strongest competitor because it handles starvation without the per-batch noise of Switch-style losses, and it reports gains on real MoEs, for example Moonlight-16B-A3B Math500 rising from 64.8% to 66.2% (Chen et al.). No reported result on quantum MoEs was found on Lacuna for φ-balancing or any balancing loss. A Switch-style batch balancing loss and an expert-orthogonality penalty modeled on $L_o$ from Guo et al., applied as a unitary-fidelity penalty, are the other two classical ports.
The key ablation is the classically routed twin. It has an identical circuit, except that the router register is measured mid-circuit and the experts are conditioned on the classical outcome. A second ablation replaces $D_{\mathrm{BD}}$ with exact numerical discord and a third replaces it with plain quantum mutual information. The mutual-information ablation isolates whether the measurement-induced part of the correlation, which is what discord adds, matters at all.
All methods share the same data, circuit ansatz, shot budget, and optimizer, with an equal grid of 8 regularization weights and 5 seeds each. Beating φ-balancing means higher mean test accuracy on MNIST-4 and Fashion-4 beyond seed spread. For the diagnostic claim, beating the baselines means that discord correlates with the quantum-versus-classical accuracy gap across runs more strongly than routing entropy or mutual information does.
Why not just…?
Why not just port φ-balancing or a load-balancing loss to the measured router outputs? These losses see only the diagonal of the router state, the probabilities of each expert index. A router in an equal superposition and a router that is a fair classical coin produce identical load statistics, so these losses cannot detect the third collapse mode by construction. They may still fix starvation as well as our regularizer does, which is a real threat to claim three. The φ-balancing arm settles this directly. If it matches the discord regularizer on accuracy, we would report discord as a diagnostic only.
Why not just replace the quantum router with a classical one? This is the most serious alternative. If the classically routed twin matches QMoE everywhere, the superposition router is unnecessary and discord merely certifies that fact. That would still be a useful negative result for quantum MoE design, but it would kill the regularizer. The proposal is built so that this outcome is measured, not avoided: the twin is run for every configuration, and the diagnostic claim is precisely that discord predicts when the twin loses.
Why not just compute exact two-qubit discord numerically instead of using the Bell-diagonal form? For one or two router qubits in simulation, exact discord is cheap: an optimization over two angles. The Bell-diagonal form earns its place only through closed-form gradients and its exact interpretation as distance to a classical-router state. The exact-discord ablation decides this. If exact discord trains as well and the projection gap is large where the router is unbalanced, we would use exact discord and keep the Bell-diagonal form only for hardware estimation.
Why not just use router entropy or mutual information? Router entropy is blind to coherence. Mutual information stays high when correlations are classical, so it cannot separate a quantum router from a classical one. The ablation compares their predictive power for the classical-twin gap against discord.
Proposed Experiments
We run the three QMoE tasks with 2 and 4 experts in noiseless and noisy simulation, logging all diagnostics every epoch. The first decisive check is the collapse census. We need seeds in which accuracy and load statistics look healthy but discord is near zero and unitary fidelity is high, and we need to show that these seeds are the ones where the classical twin matches the quantum model. The second is predictive. Across all runs and noise levels, we compute the rank correlation between final discord and the quantum-minus-classical accuracy gap, compared against routing entropy and mutual information. The third tests the regularizer against φ-balancing under matched tuning.
Before trusting any hardware estimate, we validate the estimator on known states. We prepare Bell-diagonal states across the tetrahedron with the 4-qubit purification circuit of Gårding et al., which reports tomographic fidelity of about 0.9 on hardware against about 99.5% in simulation, worst near the vertices. We then check that $D_{\mathrm{BD}}$ recovers the known discord within that error. We report exactly where near the vertices, and therefore where near fully collapsed routers, hardware noise makes the diagnostic unreliable. The final check is to measure $D_{\mathrm{BD}}$ on a trained 2-expert QMoE executed on hardware. Noise is expected to be the main confound in this run.
References
- QMoE: A Quantum Mixture of Experts Framework for Scalable Quantum Neural Networks
- Bell Diagonal and Werner State Generation: Entanglement, Non-Locality, Steering and Discord on the IBM Quantum Computer
- Advancing Expert Specialization for Better MoE
- Mixture of Experts with Soft Nearest Neighbor Loss: Resolving Expert Collapse via Representation Disentanglement
- Quantum correlations of two qubits interacting with a macroscopic medium
- Quantum Discord, Thermal Discord, and Entropy Generation in the Minimum Error Discrimination Strategy
- Entanglement as the cross-symmetric part of quantum discord
- Quantifying the unextendibility of entanglement*
- $\phi$-Balancing for Mixture-of-Experts Training