Normalize Over the Set, Not the Token: Keeping Relative User Power in Neural MIMO Detectors

Neural networks train well when their inputs and hidden activations sit in a narrow numerical range, so almost every architecture rescales them. It typically does this per feature (Z-score standardization, batch normalization) or per token (layer normalization). Wireless signals do not fit that habit. How strong one user's signal is compared with another's is not a nuisance to remove; it is the information a receiver needs to decide whom to decode first. In classical single-antenna analysis this is explicit: users are sorted by the scalar gain $|h_k|^2$, and the whole successive-cancellation strategy of non-orthogonal multiple access (NOMA, where users share the same time and frequency and are separated by power) follows from that ordering.

Our claim is that attention-based neural detectors for multi-antenna, multi-user systems (MIMO, multiple-input multiple-output) quietly throw this ordering away. The usual design builds one token per user and normalizes each token on its own, so every user reaches the network at the same scale. We propose a different rule. Normalize only over the symmetry the physics actually has, which is one shared scale for the whole received problem. In practice this means dividing all user tokens and antenna streams by a single shared statistic and feeding its logarithm back as side information. Relative gains between users and antennas survive, and optimization stays well-conditioned. We expect this change to matter most in exactly the near-far, strongly power-unequal regime where single-antenna NOMA theory says the ordering matters most.

Core Claim

Consider the linear model $\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{n}$, with $\mathbf{H} \in \mathbb{C}^{N_r \times K}$ the channel from $K$ single-antenna users to $N_r$ receive antennas. Scaling $\mathbf{y}$, $\mathbf{H}$ and the noise standard deviation by one common positive constant leaves the detection problem unchanged. Scaling individual users' columns or individual antennas differently does not: it changes who is strong, who should be cancelled first, and how much interference each stream sees.

A normalization layer is therefore safe only if the set of transformations it is invariant to is contained in that one-parameter group. Per-feature standardization and per-token layer normalization are invariant to a much larger group, so they erase information the task needs. The fix we test is set-level normalization. For hidden user tokens $z_1,\dots,z_K \in \mathbb{R}^d$ belonging to one received block,

$$ \tilde z_k = \frac{z_k}{s}, \qquad s = \sqrt{\tfrac{1}{Kd}\sum_{j=1}^{K}\lVert z_j\rVert^2 + \epsilon}, $$

and $\log s$ is appended to every token. Because $s$ is a symmetric function of the whole set, the layer keeps the permutation equivariance (treating users as an unordered set) that attention detectors rely on. Because $s$ is an average rather than a sum, it does not drift with $K$ or $N_r$. That should matter when a detector is trained at one array size and deployed at another.

The prediction is falsifiable. Per-token normalization should cost little when users arrive at similar power and increasingly more as the power spread between users grows. Set-level normalization should stay flat across that spread.

Where This Sits

Two strands of prior work motivate the idea.

On the scaling side, the file's general-purpose sources argue that rescaling is needed to stop large-magnitude inputs from dominating gradients (steelmaking study). FD-Loss shows that per-feature $\ell_2$ normalization is what makes a decorrelation penalty stable on heterogeneous-scale tabular data. Both treat magnitude as something to neutralize.

The strongest evidence that this backfires in wireless comes from Normalization-Free Knowledge Distillation for Variable-Bit CSI Feedback. That paper compresses channel measurements for feedback on DeepMIMO O1 and finds that Z-score-normalized distillation performs worse than no distillation at all at low bit rates, because it destroys path-loss and array-gain information. Its remedy is to drop normalization entirely inside the distillation loss.

We take the opposite route from both camps. We keep normalization but restrict its invariance to the physically correct group, and we move the question from a loss function into the detector's hidden layers.

On the antenna side, Optimal Decoding Order and Power Allocation in Downlink NOMA derives decoding-order results that hold only because single-antenna gains are totally ordered. It flags multi-antenna extension as blocked by the loss of that ordering. We supply the premise; the paper's result does not depend on neural networks.

RE-MIMO supplies the architecture we modify. It is a recurrent, permutation-equivariant detector that uses self-attention over user tokens and a physics-based likelihood module, and it generalizes zero-shot across user counts. It deliberately does not order users. Our question is whether the relative power information that would let it act like an ordered cascade actually survives its normalization layers. The file does not describe RE-MIMO's normalization, so our first step is to audit the released configuration rather than assume it.

DeepJSCC-MIMO is the closest positive precedent. It feeds a heatmap of per-subchannel noise levels to a vision transformer so the network can learn water-filling-like power allocation instead of losing the scale. That is side-information conditioning at the input, which is one half of our layer (the $\log s$ token). We ask whether internal normalization undoes it.

Research Program

The work has three parts.

Audit. We take the RE-MIMO design and a generic transformer detector with pre-norm per-token layer normalization. We train each on multi-user uplink with controlled power disparity and probe the hidden tokens. A linear probe should predict each user's rank in received power. If per-token normalization erases the information, probe accuracy should collapse toward chance as the disparity grows while detection error rises.

Intervention. We swap only the normalization layers in the same networks: set-level normalization as above, with and without the $\log s$ token. Parameter count and training schedule stay fixed.

Transfer. We test the size-consistency argument. Models trained at $N_r = 4$, $K \in \{4,6,8\}$ are evaluated zero-shot at unseen $N_r$ and $K$, including the degenerate $N_r = 1$ single-antenna case. That case links back to the scalar-ordering theory: a detector that has kept relative power should reproduce the single-antenna decoding-order behaviour there.

As a secondary check of generality, we insert set-level normalization into the teacher-student comparison of the DeepMIMO CSI feedback setting. This asks whether scale-restricted normalization can match normalization-free distillation, rather than normalization having to be abandoned.

Baselines

Do-nothing reference. The same detector with every normalization layer removed and raw received samples as input. It isolates whether normalization of any kind is needed. We expect it to match set-level normalization at modest dynamic range and to become unstable or slow to converge when the power spread reaches 30 dB. It gets the same data, optimizer and learning-rate sweep as every other variant, with gradient clipping allowed so it is not handicapped.

Standard ML recipes. Input Z-score standardization per antenna feature, and per-token layer normalization inside the attention blocks. These are what a practitioner would reach for, and what the audit targets.

Strongest existing competitor. RE-MIMO as released, because it is the only neural detector in the file with built-in permutation equivariance and demonstrated zero-shot transfer across user counts. We run it with its published configuration and with a matched retuning budget on our data. We found no reported detection-error figure for RE-MIMO in the file ("no reported result found on Lacuna for RE-MIMO symbol error rate"), so the comparison is against our own reproduction.

Classical references. Linear MMSE detection, and MMSE with successive interference cancellation where users are ordered by post-equalization SNR. The second is the natural multi-antenna analogue of the single-antenna NOMA cascade.

Key-ingredient ablations. Set-level normalization without the $\log s$ token, and per-token normalization with a per-token log-scale token added back. Both keep the architecture fixed and isolate which piece carries the effect.

What counts as beating the baselines. Set-level normalization must match per-token normalization within noise at 0 dB disparity. At 20 to 30 dB disparity it must reduce symbol error rate relative to both per-token normalization and the unnormalized network, while converging at least as fast as the per-token variant.

Secondary check on CSI feedback. The reference is NF-KD's reported −12.87 dB normalized mean-squared error (NMSE) at 256 bits, against −11.98 dB for its baseline, on DeepMIMO O1 (NF-KD). Set-level normalized distillation should reach at least that figure, and should clearly beat Z-score distillation at low bit rates.

Why not just…?

Why not just drop normalization, as NF-KD does? This is the most serious threat, because NF-KD shows normalization-free training works on its task. That task has a single-user channel with a bounded dynamic range. Multi-user detection with near-far users multiplies the dynamic range, and the file's scaling literature predicts that the largest-magnitude inputs will then dominate the gradients. The do-nothing baseline swept across power disparity settles this. If it stays competitive and stable at 30 dB disparity, our contribution shrinks to "don't normalize per token," and we will report it that way.

Why not just divide by the noise variance and feed SNR as an input, as DeepJSCC-MIMO does with its heatmap? At the input this is nearly our proposal, since dividing by one noise scalar respects the correct symmetry. The difference is inside the network: DeepJSCC-MIMO conditions its input, but per-token normalization in later layers can still equalize users. The ablation "input noise-normalization plus internal per-token normalization" against "input noise-normalization plus set-level normalization" shows whether input conditioning alone suffices. It also covers settings where the noise variance is only estimated.

Why not just equalize with MMSE first and run a single-antenna network per stream? That recovers the scalar world, but it inherits noise enhancement on ill-conditioned channels. Space-time DeepJSCC found a linear MMSE equalizer insufficient on its own and added a residual network to refine its output, and RE-MIMO notes that matrix-inversion cost grows cubically with array size. We include MMSE and MMSE with successive cancellation as baselines, and we report the correlated-channel results where conditioning is poor.

Proposed Experiments

Channels. We use Rayleigh fading with and without exponential spatial correlation, plus DeepMIMO O1 ray-traced channels. In DeepMIMO, user positions produce natural path-loss spreads instead of synthetic ones. On the synthetic channels, per-user power offsets are drawn uniformly up to 0, 10, 20 and 30 dB. We use QPSK and 16-QAM, with SNR swept from 0 to 25 dB.

Metrics. Symbol error rate against SNR at each disparity level, training steps to a fixed error, the linear-probe accuracy for power rank, and zero-shot symbol error rate at unseen $K$ and $N_r$ (including $N_r = 1$). For the CSI feedback check we report NMSE against feedback bits on the O1 6:2:2 split.

Decisive experiment. The disparity sweep with everything else held fixed. The hypothesis fails if the per-token and set-level curves do not separate as disparity grows. It is confirmed if they separate, and the probe shows power-rank information disappearing only under per-token normalization. A second decisive check is the transfer to $N_r = 1$: a set-normalized detector there should decode strong-first, matching the single-antenna NOMA prescription, while a per-token detector should not.