When Can a Trained Channel Estimator Beat the Pilot-Contamination Ceiling?

Large cellular antenna arrays (massive MIMO) hit a hard limit called pilot contamination. Users in neighbouring cells must reuse the same short training signal, or pilot, so the base station sees the desired user's channel added to an interferer's channel. If the two channels have the same spatial covariance, meaning the same average angular spread of energy, then every linear estimator is stuck. The best linear estimator, linear minimum mean-square error (LMMSE), cannot tell the two channels apart. Neural estimators are now trained to clean up these mixtures, but nobody can say whether they can do better than linear methods in this hardest case, or how much data and training they need. The Gaussian equivalence principle is a body of high-dimensional learning theory showing that many trained models behave as if their data were Gaussian with the same mean and covariance. Taken at face value, it says neural estimators should hit exactly the same wall as LMMSE.

We propose to resolve this tension with an exact theory of training dynamics. Our claim is that a two-layer neural channel estimator trained by stochastic gradient descent first gets stuck on a plateau whose error equals the LMMSE ceiling; this is the regime where Gaussian equivalence holds. Then, if the training data contain enough samples per antenna and the channels carry non-Gaussian structure, the network escapes the plateau by aligning its weights with the propagation directions. We will derive the low-dimensional differential equations that govern this escape, predict the sample-size threshold and plateau length as functions of antenna count, SNR and interference power, and turn the theory into a practical diagnostic. The diagnostic retrains each estimator on a Gaussian surrogate of the real channel data. The resulting "surrogate gap" measures how much decontamination gain is available beyond second-order statistics at all.

Core Claim

The research file sets out the tension. On the wireless side, A Coordinated Approach to Channel Estimation in Large-scale Multiple-antenna Systems proves that covariance-aided estimation removes contamination only when users sharing a pilot have non-overlapping angle-of-arrival supports. Massive MIMO with Imperfect Channel Covariance Information shows that even this route is limited by how hard it is to estimate covariance. On the learning side, Universality of empirical risk minimization proves that risk depends only on the first two moments of the features when the parameters are spread across coordinates. Learning curves of generic features maps for realistic datasets with a teacher-student model confirms this on real images for squared loss.

There is a third piece. Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed shows that two-layer networks trained by one-pass SGD escape kernel-like behaviour on mixtures whose structure is hidden from second moments. Fixed-feature methods cannot. That paper motivates our mechanism.

Put together, these results do not yield a contradiction. They yield a phase diagram. Gaussian equivalence governs estimators with fixed or barely moving features, and the early phase of feature-learning networks. Escaping the LMMSE ceiling requires the network to break symmetry in a way that the macroscopic dynamics can track. No paper in the file states this phase diagram for pilot contamination, predicts where its boundaries lie, or tests it.

A Tractable Contamination Model with Matched Moments

To isolate the question, we build a "matched-moment" contamination model. In it, the desired and interfering channels have identical mean and covariance, so the genie-covariance LMMSE estimator gains nothing from spatial filtering. The two channels differ only in higher-order structure.

The desired user is near the array and propagation-sparse. Its channel is $h_d = g\,a(\theta_k)$, where $a(\theta)$ is the array response vector (the "steering vector") for arrival angle $\theta$, $g$ is a complex Gaussian gain, and $k$ is drawn uniformly from $K$ dominant directions. The interferer is far away and sees rich scattering. Its channel $h_i$ is complex Gaussian with covariance $\frac{1}{K}\sum_k a(\theta_k)a(\theta_k)^H$, which equals the covariance of $h_d$. The base station observes $y = h_d + \sqrt{\beta}\,h_i + n$ with interference power ratio $\beta$ and noise $n$.

Conditional on $k$, the observation $y$ is Gaussian. This property makes the analysis tractable: $y$ is a zero-mean Gaussian mixture whose components differ only in covariance. It is a covariance-only analogue of the mixtures in Refinetti et al. As a result, the expectations that drive SGD reduce to Gaussian integrals over a handful of overlaps. The Bayes-optimal estimator is computable in this model, so the gap between the LMMSE ceiling and the Bayes limit is known exactly.

The model is also physically meaningful. It describes a near line-of-sight user contaminated by a diffuse far-cell user within the same angular sector. This overlapping-angle-of-arrival case is exactly the one the wireless literature identifies as unsolved.

Research Program

The estimator is a two-layer network $f(y)=\sum_{j=1}^{p} v_j\,\sigma(w_j^\top \tilde y)$. Here $\tilde y$ is the real representation of $y$, the $w_j$ are first-layer weights, and the $v_j$ are $M$-dimensional output vectors. It is trained by one-pass SGD on squared error against $h_d$. Following From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks, we track three sets of order parameters, which are scalar summaries of the weights. The first is the student–direction overlaps $m_{jk}=\frac{1}{M}w_j^\top a(\theta_k)$. The second is the student–student overlaps $Q_{jl}=\frac{1}{M}w_j^\top w_l$. The third is the corresponding projections of the $v_j$. We will derive the closed ordinary differential equations for these quantities in the limit of many antennas with the learning rate scaled appropriately.

The theory should deliver three results. First, the uninformed fixed point $m_{jk}=0$ has test error equal to the matched-moment LMMSE error; this is the Gaussian-equivalence plateau. Second, linearising the equations around that fixed point should show overlap growth proportional to the current overlap, starting from random-initialisation scale $1/\sqrt{M}$. That would give an escape time of order $\log M$ in rescaled time, or about $M\log M$ fresh samples. The proportional growth is what we expect from a target whose non-Gaussian signal first appears at second order in each direction. Deriving it, and finding its dependence on SNR and $\beta$, is the central technical task. Third, we will compute the post-escape error and compare it with the Bayes limit.

For the lazy regime, where the first-layer weights are effectively frozen, we will reuse the random-feature gradient-flow solution of Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model. That solution linearises the activation via Gaussian equivalence and solves the training trajectory exactly. Applying it here should show the random-feature estimator pinned at the LMMSE ceiling for every fixed ratio of samples to antennas. The bridge to Saeed's delocalization condition is a point we will make explicit. Steering vectors are spread out across antenna coordinates but concentrated in beamspace, the representation indexed by angle rather than antenna. Gaussian equivalence can therefore hold or fail depending on whether the learned weights concentrate in the basis in which the channel is sparse. We will test this prediction by tracking the beamspace localisation of the learned $w_j$.

The second strand moves to realistic channels. For channels from QuaDRiGa and the 3GPP Urban Macro-Cell model, the multi-cell generators used by Addressing Pilot Contamination in Channel Estimation with Variational Autoencoders, we construct Gaussian surrogates with exactly matched per-user mean and covariance. We then train each estimator on the real channels and on the surrogates. The difference in test NMSE is the surrogate gap. For the lazy estimators it should be zero up to finite-size effects. For feature-learning estimators it should be positive only past the predicted threshold in samples per antenna. This gives practitioners a cheap test of whether a learned decontaminator is doing anything a covariance estimator could not.

Baselines

The naive reference is the least-squares estimate, which simply despreads the pilot and passes $y$ through. It sets the uncorrected contamination floor.

The most important analytical reference is LMMSE with genie per-user covariances, as in Yin et al. On the matched-moment model this is the two-moment ceiling by construction. On realistic channels it is the strongest linear estimator possible. We will also run its practical version, the "Via Q" covariance estimator of Björnson, Sanguinetti and Debbah. That paper reports on a 7-cell, 100-antenna one-ring layout that Via Q reaches 95% of perfect-covariance performance with 100 extra pilots, where direct estimation needs 250. We give it the same pilot budget that the neural estimators see as training data.

The strongest learned competitor is the multi-cell conditionally Gaussian VAE of Kasibovic et al., which outputs per-observation means and covariances for a conditional LMMSE step. Because a conditionally Gaussian model is a Gaussian mixture, it can in principle exploit exactly the non-Gaussian structure our theory identifies. That makes it the real benchmark for the "beyond the ceiling" regime. The file contains no reported NMSE number for it on the 3GPP or QuaDRiGa setups, so we will retrain it ourselves using its published architecture.

We also include the three-layer MLP denoiser of Based on Deep Neural Networks: A Machine Learning-Assisted Channel Estimation Method for MIMO Systems. It reports NMSE of −34.46 dB against −30.91 dB for MMSE at 30 dB SNR on 4×4 COST 2100 channels. That setting is small and only weakly contaminated, so we treat the number as a sanity anchor rather than a target.

All estimators share identical training channels, SNR grid, parameter count where applicable, and a matched hyperparameter search budget. The theory-matched two-layer network is not expected to beat the VAE outright. Two outcomes would count as success. The first is that the predicted plateau value, escape threshold and post-escape NMSE match simulation within 0.5 dB across antenna counts from 32 to 256. The second is that the surrogate gap, as predicted in advance, separates the estimators that beat genie LMMSE from those that do not.

Two ablations isolate the key ingredient. The first is the same network trained in lazy (kernel-like) scaling instead of mean-field scaling. The second is the same network trained on Gaussian-surrogate data. Feature learning on non-Gaussian channels should be the only combination that leaves the plateau.

Why not just…?

Why not just use LMMSE with well-estimated covariances, since it is Bayes-optimal for Gaussian channels? On the matched-moment model, LMMSE is provably stuck: the users' covariances coincide, so Yin et al.'s separation argument gives it nothing. Bayes-optimal error there is strictly lower and computable, so the gap is measured, not assumed. On real channels the surrogate gap answers the question directly. If the gap is near zero for QuaDRiGa, then LMMSE with Via Q covariances is the right answer and we will say so. That would be a useful negative result for the deep-learning channel-estimation literature.

Why not just use the conditionally Gaussian VAE, which already works empirically? This is the real threat to any claim of practical improvement, and we do not claim to beat it. What it lacks is a prediction. Kasibovic et al. report that it approaches genie-covariance performance when angles of arrival are separated, but they give no account of when it can go further, nor of how much data it needs. Our theory predicts a threshold in samples per antenna and an $M\log M$ escape time. Running the VAE on the matched-moment model across antenna counts will show whether its learned latent structure crosses the ceiling at the same threshold. If it does, the dynamics explain it. If it beats the ceiling below the threshold, our two-layer theory is missing a mechanism, and that would also be informative.

Why not just use random features or kernel ridge regression, where Gaussian equivalence is already solved exactly? Because the theory predicts that these estimators cannot leave the plateau. This is the kernel trap of Refinetti et al., transposed to estimation. The lazy-scaling ablation tests it: if random features at a fixed ratio of samples to antennas ever beat matched-moment LMMSE by more than finite-size noise, the core claim is wrong.

Why not just run large simulation sweeps instead of deriving equations? Sweeps alone cannot separate a slow plateau from a permanent ceiling at the training budgets used in practice. They also do not extrapolate from 32 to 256 antennas. The equations predict the plateau length as a function of $M$, SNR and $\beta$ before any sweep is run. That turns each simulation into a falsification test rather than a curve to fit.

Decisive Checks

The first check is plateau identity. In the matched-moment model at 64, 128 and 256 antennas, the test NMSE of the feature-learning network during its plateau, and of the random-feature network at all times, must equal matched-moment LMMSE within finite-size fluctuations.

The second check is escape scaling. The number of SGD samples needed to leave the plateau, divided by $M\log M$, should collapse onto one curve across antenna counts. It should shift with SNR and $\beta$ as the linearised equations predict.

The third check is the surrogate gap on QuaDRiGa and 3GPP Urban Macro-Cell multi-cell data. The gap should be zero for the lazy estimators and for any estimator trained below the predicted threshold. It should be positive only for feature-learning estimators trained above it.

The fourth check is beamspace localisation. Escape should coincide with the learned first-layer weights concentrating in beamspace, measured by the inverse participation ratio of each $w_j$ in the angle domain. That would link the breakdown of Gaussian equivalence to the delocalization condition of Saeed's universality theorem.

If the second and third checks both fail, learned decontamination in this regime is effectively a covariance estimator, and the wireless community should invest in covariance estimation instead.