When the Expert Choice Costs Bits: Mixture-of-Experts CSI Feedback Where Collapse Wastes Uplink Bandwidth

A phone served by a many-antenna base station has to describe its downlink channel back to the station. That description is the channel state information (CSI), a large complex matrix over antennas and subcarriers, and the uplink bits it consumes eat into the gains the antennas were supposed to provide. Neural autoencoders compress this matrix well within one propagation environment. They degrade in unfamiliar ones, and they usually need separate weights for each bit budget. A natural fix is a mixture of experts (MoE), in which several specialized decoders sit at the base station and a small selector picks one per channel. In most MoE work the selector's decision is free internal computation. Here we propose putting the selector on the phone, so that the chosen expert's index is sent as part of the feedback payload. That index then costs real uplink bits.

This changes what "expert collapse" means. Collapse is the known MoE failure in which the selector sends everything to one expert, or the experts become near-copies of each other. Once the index is transmitted, collapse has a direct price in bits. If the selector starves most experts, the index carries almost no information. If the experts are homogenized, the index carries information that buys no reconstruction accuracy. In both cases the $\log_2 E$ index bits, where $E$ is the number of experts, would have been better spent on the compressed channel itself. Our central claim is that, at a fixed total bit budget, an MoE decoder beats a dense autoencoder only if its anti-collapse training produces an index that is both high-entropy and useful. We further claim that population-level balancing combined with an explicit specialization penalty achieves this where standard per-batch balancing losses do not. The proposal turns an MoE training pathology into a quantity we can measure: NMSE (normalized mean squared error, the standard reconstruction metric) gained per index bit.

Core Claim

The total feedback budget is $B$ bits. The payload splits into $\log_2 E$ bits of expert index and $b_z = B - \log_2 E$ bits of quantized latent code. We define the value of the index as

$$\Delta(E, B) = \mathrm{NMSE}_{\text{dense}}(B) - \mathrm{NMSE}_{\text{MoE}}(E, B),$$

where both NMSE values are in dB. $\mathrm{NMSE}_{\text{dense}}(B)$ is the error of a single decoder that receives all $B$ bits as latent code. $\mathrm{NMSE}_{\text{MoE}}(E, B)$ is the error of the $E$-expert model under the split above. A positive $\Delta$ means routing is worth its bits.

We predict that $\Delta$ is negative when routing is left unregularized, because the selector collapses and the index entropy falls well below $\log_2 E$. We predict $\Delta$ is roughly zero or negative under a Switch-style per-batch balancing loss, because the index entropy is high but the experts converge to similar functions. We predict $\Delta$ is clearly positive only when balancing is imposed over the whole population of channels and expert outputs are pushed apart. The effect should be largest when the training pool mixes heterogeneous propagation conditions and at small $B$, where one well-spent index bit matters most relative to the latent.

Why This Intersection, and What Is Already There

The CSI literature motivates the need for specialization. AdapCsiNet generates base-station decoder weights from a scene graph of room walls with a hypernetwork, reporting 2.7 dB NMSE gain at compression ratio 1/24 and outperforming coarse model-switching. EG-CsiNet aligns propagation paths with SVD and DFT codebooks before compression and generalizes across 90 unseen WAIR-D environments. Both show that one monolithic decoder is a poor fit for diverse multipath. Both also require side information, either a scene map or phone-side path decomposition, that we do not assume. We instead let the channel itself choose a discrete decoder and pay for that choice in bits.

The closest work combining experts and wireless is Team-DMoE. It mixes dense power-control networks using a gate fed with a noise estimate. Its experts are hand-assigned to noise regimes, it does not compress CSI, and it never faces collapse because nothing is learned end to end. The position paper Against the Monolithic Wireless World Model argues for composable specialized components in 6G but offers no routing mechanism or experiments. Our proposal differs from all three in the same way: routing is learned, sparse, and transmitted over the air.

The MoE literature supplies the method and also constrains it. Advancing Expert Specialization for Better MoE and OMoE document that balancing losses can drive experts to near-identical representations, up to 99% similarity. The first adds an orthogonality loss on expert outputs and a variance loss on routing scores. The second uses orthogonal projection steps in the optimizer. φ-Balancing replaces per-batch uniform quotas with a population-level entropy potential that tracks expert loads through a moving average. This matters in our setting because CSI mini-batches are often drawn from one site and are legitimately unbalanced. BASE Layers enforces exact balance by solving an assignment problem during training. The same paper constrains our design: that balance cannot be enforced at test time, when inputs arrive one at a time. Because our selector runs per channel on the phone, we need a method that leaves the selector balanced at inference without any batch-level solver. That requirement favors φ-balancing over BASE-style assignment.

One connection between the two fields is exact. The entropy potential in φ-balancing is the Shannon entropy of the expert-load distribution. For a transmitted index, that same quantity is the information content of the index bits. In our setting the balancing term is therefore a rate term, not only a hardware heuristic.

Research Program

The phone runs a light selector: a two-layer network acting on the magnitude of the angular-delay transform of its channel estimate, which costs one 2D-DFT and a small matrix product. It also runs a shared linear or shallow encoder that produces $b_z$ latent bits. The base station holds $E \in \{2, 4, 8, 16\}$ expert decoders. Each decoder is a feed-forward CsiNet-style network, and the transmitted index activates exactly one of them.

Training is end to end. Gradients pass through the discrete choice with a straight-through Gumbel estimator. The loss combines reconstruction NMSE, a φ-balancing potential on the moving-average expert load, and the orthogonality loss of Guo et al. applied to the outputs of different experts on the same input. We keep reconstructions unnormalized, following NF-KD, because absolute channel magnitude carries path-loss information that a selector could use.

The analytical side tracks four quantities throughout training. These are the empirical index entropy, the mutual information between the index and the ground-truth scenario label, an expert-similarity score on shared inputs, and $\Delta(E, B)$. Together they separate starvation from homogenization. A secondary analysis tests the entropy-coded variant, where the index is transmitted at its empirical entropy rather than at $\log_2 E$ bits, as in the entropy coding of DeepCMC. In that regime collapse becomes cheap in bits rather than wasteful, and the right balancing strength should follow a rate-distortion trade-off instead of a uniform target. We will test whether the learned operating point matches that prediction.

Baselines

The naive reference point is a dense autoencoder with the same phone-side encoder and one base-station decoder. It receives all $B$ bits as latent, and its decoder parameter count is matched to the total parameters of the expert bank. Being deliberately generous on parameters rules out the explanation that the MoE wins only because it has more parameters.

The strongest existing dense method is NF-KD, a normalization-free distilled variable-bit autoencoder. It reports −12.87 dB NMSE at 256 bits on DeepMIMO O1 at 140 GHz, against −11.98 dB for its baseline, and remains robust down to 32 bits. We rerun it on the same split, 6:2:2 as in that paper, with the same number of tuning trials and GPU-hours as our model. Beating it means lower NMSE at an equal total bit count on the single-scenario O1 benchmark. It also means a larger margin on the mixed-scenario pool.

The strongest non-neural reference is site-specific subspace inference from Beyond Type-II. It reports roughly 99% signal-capture efficiency relative to Type-II in simpler scenarios and the best effective spectral efficiency among codebook schemes. We will compare against it on spectral efficiency and capture efficiency on the DeepMIMO scenarios it used.

For one-sided designs we include CSI-PPPNet. It runs a random projection on the phone and 10 to 20 denoising iterations at the base station, with 175.2K parameters across five compression ratios. We report NMSE against base-station compute on COST 2100.

The ablations isolate the key ingredient. We compare no balancing, a Switch-style auxiliary loss, BASE-style assignment during training with greedy selection at test time, φ-balancing alone, and φ-balancing combined with the orthogonality loss. All variants run at identical $E$ and $B$ with an equal search budget over the balancing weight.

Why not just…?

Why not just give all the bits to one good dense decoder? This is the null hypothesis, and $\Delta(E, B)$ tests it directly. The evidence that it falls short is indirect. AdapCsiNet and EG-CsiNet both report multi-dB losses for monolithic decoders across environments. Neither compares against a discrete index at equal bits. If $\Delta \le 0$ everywhere, including on the mixed pool at 32 to 64 bits, we will report that routing is not worth its bits.

Why not just cluster the channels offline and train one decoder per cluster? This is the real threat. We run k-means on angular-delay power profiles with $E$ clusters, send the cluster index, and train one decoder per cluster, all at equal bits. AdapCsiNet's win over coarse indoor/outdoor switching does not settle the question, because clustering on the channel itself is finer-grained than a two-way environment label. If offline clustering matches learned routing, then the contribution reduces to finding the index split, and the anti-collapse machinery is unnecessary. We would then state that result plainly.

Why not just route at the base station from the received latent, so the index costs nothing? This keeps all $B$ bits for the latent. However, the selector then sees only a compressed, lossy view of the channel, which is the open question raised by one-sided designs. We run this variant as an ablation. If it matches phone-side routing, the transmitted-index framing is unnecessary. If phone-side routing wins at small $B$, the index is carrying information the latent cannot.

Why not just use a standard Switch-style balancing loss? Advancing Expert Specialization and OMoE document homogenization under such losses on text, and φ-Balancing wins over them in more than 80% of 21 setups. None of this has been tested on physical-layer regression. Our ablation measures high index entropy paired with low index-scenario mutual information and high expert similarity, which is the signature of homogenization.

Proposed Experiments

The primary testbed is DeepMIMO. We regenerate asu_campus_3p5, boston5g_3p5, O1_28, and O1B_28 under one common array and subcarrier configuration and pool them. The pool mixes rich-multipath, urban, line-of-sight, and blocked conditions within one training set. We also run the O1 140 GHz configuration (64 base-station antennas, 128 subcarriers) alone, so that we have the NF-KD comparison point. Bit budgets span 32, 64, 128, and 256.

Generalization is tested on WAIR-D, holding out unseen city environments in the manner of EG-CsiNet. That paper reports more than 3.5 dB improvement over UniversalNet+, but the file contains no absolute NMSE, so no reported result was found on Lacuna for EG-CsiNet's absolute WAIR-D NMSE. We will therefore rerun EG-CsiNet ourselves where feasible. COST 2100 indoor and outdoor data give a second two-mode check.

The metrics are NMSE, cosine similarity of the reconstructed beamformer, spectral efficiency at 20 dB SNR, phone-side FLOPs, and the four collapse diagnostics described in the research program.

The decisive check is a curve of $\Delta(E, B)$ against $B$ for each balancing variant on the mixed pool, with the clustering baseline plotted on the same axes. The proposal succeeds if φ-balancing combined with orthogonality gives positive $\Delta$ at small $B$ that exceeds both the clustering baseline and the base-station-routing ablation, while the unbalanced and Switch variants show the predicted starvation and homogenization signatures. The proposal fails cleanly if clustering or the dense decoder matches it.