Federated learning trains one model across many phones or sensors without collecting their data. Over-the-air aggregation makes this cheap: every device transmits its gradient at the same moment on the same frequency, and the radio channel adds the signals for free. A multi-antenna base station can then average away much of the noise in that sum. The usual reason is spatial error averaging. The wanted signal adds up coherently across antennas, so its power grows as the square of the antenna count $N$. Errors that arise independently at each antenna add up incoherently, so their power grows only as $N$.
What these systems still lack is a way to pick the learning rate without running many full training trials over a scarce radio link. The natural tool is hypergradient descent, which nudges the learning rate up or down according to whether two consecutive gradients point the same way. We argue that this tool fails over the air for a specific, fixable reason. The inner product it relies on is biased by errors that persist from one round to the next. Spatial averaging shrinks those errors but does not remove them from the inner product. Our fix is simple. We split the antenna array into two disjoint halves and take the inner product across halves and across rounds. This cancels every persistent error that originates at the base station. What remains is a bias floor set only by the devices' own hardware.
Core Claim
Hypergradient descent, as proposed in Online Learning Rate Adaptation with Hypergradient Descent, updates the learning rate as $\alpha_t = \alpha_{t-1} + \beta\, \hat g_{t-1}\cdot \hat g_{t-2}$. Here $\hat g$ is the gradient the server actually receives and $\beta$ is a small "hyper" step size. Over the air, the received aggregate at round $t$ from a set of antennas $S$ is well described as
$$\hat g_t^{S} = g_t + e_t^{S} + u_t .$$
In this decomposition, $g_t$ is the true averaged gradient. The term $e_t^{S}$ collects errors created at the base station: receiver noise, low-resolution converter distortion, and misalignment from imperfect channel estimates on the antennas in $S$. The term $u_t$ collects distortion created inside the devices' own transmitters. Taking expectations of the hypergradient gives
$$\mathbb{E}[\hat g_t\cdot \hat g_{t-1}] = g_t\cdot g_{t-1} + \mathbb{E}[e_t\cdot e_{t-1}] + \mathbb{E}[u_t\cdot u_{t-1}] .$$
Thermal noise is fresh each round, so it contributes nothing to this expectation, only variance. Two other error sources do persist across rounds.
The first is channel-estimation error. It persists whenever the same estimate is reused within a coherence block, the period over which the channel stays roughly constant. The second is quantization distortion. A quantizer is a deterministic function of its input, so when consecutive gradients are similar, their quantization errors are similar too.
Both sources make the middle term positive. That term sums over all $d$ model coordinates. A per-coordinate correlation $\rho$ at per-coordinate error variance $\sigma^2/N$ therefore yields a bias of order $d\rho\sigma^2/N$. Late in training, the true term $g_t\cdot g_{t-1}$ shrinks toward zero while this bias does not. The learning rate then drifts upward, which offers a concrete explanation for the late-stage instability already flagged for hypergradient methods.
Errors on disjoint antennas are independent: channel-estimation errors are per-antenna, and so are converter errors. The cross-half estimator $\hat g_t^{A}\cdot \hat g_{t-1}^{B}$, with $A$ and $B$ disjoint halves of the array, therefore removes the middle term entirely. The cost is a doubling of the per-half variance. The device-side term $u_t$ rides on each user's own channel, so it is common to both halves and survives.
This matches the asymmetry in Massive MIMO Systems with Non-Ideal Hardware: Energy Efficiency, Estimation, and Capacity Limits. That paper models distortion power as proportional to signal power, $\kappa^2 P$, and shows that base-station flaws vanish as $N$ grows while user-hardware flaws do not. The new prediction is that the residual hypergradient bias after splitting scales with the device distortion level $\kappa^{\mathrm{UE}}$ divided by the number of devices $K$. Devices' distortions are independent of each other, so they average over $K$. The bias does not shrink with $N$ at all.
Research Program
The work has three parts.
The first part is analysis. We derive the bias and variance of the full-array and split-array hypergradients under the impairment models in the file. For channel estimates we use the Gauss–Markov error of Over-the-Air Computation in OFDM Systems with Imperfect Channel State Information. That paper shows the aggregation error plateaus at an MSE of about 0.08 with one antenna and estimate-error variance 0.2, and falls by nearly two orders of magnitude at $N=100$. For coarse converters we use the Bussgang linearization $\mathcal{Q}(y)=Ay+\eta_q$ of Throughput Analysis of Massive MIMO Uplink With Low-Resolution ADCs. For device distortion we use the $\kappa^2 P$ model. Chen et al. supply the noise model; their concern is the magnitude of aggregation error. Ours is its correlation across rounds, which they do not study because they treat aggregation as a one-shot computation.
The second part is the algorithm. It is server-side hypergradient adaptation of the learning rate on top of the server optimizer used in Adaptive Federated Learning Over the Air. That work, called ADOTA-FL, applies Adam and AdaGrad at the server to noisy over-the-air gradients and proves convergence under heavy-tailed noise. However, it keeps the base learning rate fixed and assumes a single effective receive antenna, so it has neither spatial averaging nor learning-rate search. We take its Adam variant as the base, add the Adam form of hypergradient descent from Baydin et al., and replace the inner product with the cross-half version. We also normalize $\beta$ by a running estimate of the hypergradient variance, which is itself computable from the two halves.
The third part is an empirical test of where the theory breaks. The Bussgang model treats distortion as uncorrelated with the signal. Under deep 1-bit clipping this is known to be only approximate, and the distortion may then correlate across antennas, which would leak bias through the split.
Baselines
The naive reference is Adam-OTA from ADOTA-FL with a single fixed learning rate taken from that paper's setting. It is run with the same devices, rounds, channel, and impairments. On CIFAR-10 with ResNet-18 and a strongly non-i.i.d. Dirichlet 0.1 partition, ADOTA-FL reports that Adam-OTA reaches 70% accuracy in about 200 rounds, while its baselines fail to reach 40% after 600 rounds. This is the strongest reported result on the task. Our method beats it if it reaches 70% in fewer rounds without any learning-rate search, or matches it from a badly chosen initial learning rate where fixed-rate Adam-OTA does not.
The strongest tuned competitor is an offline grid over the learning rate for Adam-OTA, run in simulation. We report it two ways. One is best-in-hindsight accuracy. The other is the total radio rounds the grid consumes, charging every trial, since search overhead is the point. BOHB, from BOHB: Robust and Efficient Hyperparameter Optimization at Scale, is the principled multi-fidelity version of this search. It combines early stopping of weak trials with a model of which settings look promising, and it reports 2.78% CIFAR-10 test error at 33 GPU days in the centralized setting. We run it with round count as the fidelity, under a total-round budget equal to a small multiple of one training run. Beating it means matching its final accuracy within one standard deviation across seeds while using a single run's rounds.
The key ablation is plain hypergradient Adam on the full-array aggregate, with the same $\beta$ and the same variance normalization. Only the split is removed. If the split-array version is not more stable than this as $N$, the converter bit width, and channel-estimate reuse are varied, the core claim fails.
A second ablation keeps the split but re-estimates the channel every round. This isolates the channel-persistence contribution from the converter-correlation contribution. The pilot overhead it incurs is charged against its round budget.
For hypergradient methods run over the air on any benchmark, no reported result was found on Lacuna.
Why not just…?
Why not just smooth the hypergradient with a moving average or clip it? Averaging lowers variance but leaves the expectation unchanged. The failure we predict is a positive bias, so a moving average would converge confidently to the wrong learning rate. The full-array ablation, run with the same smoothing, tests this directly. If smoothing alone removes the upward drift, the bias is smaller than our analysis says, and we will report that.
Why not just re-estimate the channel every round, or correlate gradients several rounds apart? Fresh channel estimates do decorrelate the estimation error, and this is a real threat. It costs pilot time on every round, though, and it leaves converter distortion correlated whenever consecutive gradients are similar. Taking the product of gradients $k$ rounds apart, with $k$ beyond the coherence time, also removes channel persistence. The price is that the true gradients themselves decorrelate over $k$ rounds, which weakens the signal that hypergradient descent needs. Both appear in the experiments under equal pilot and round accounting. If either matches the split under 1-bit and 3-bit converters, we would recommend it as the simpler option.
Why not just use a reliable digital uplink and tune offline with BOHB? Digital uplinks need a separate time or frequency slot per device, so latency grows with $K$. Over-the-air aggregation is constant in $K$, which is ADOTA-FL's motivating argument. Separately, BOHB remains an outer loop of many trials. In federated learning each trial costs radio rounds and device energy rather than idle GPU time. The offline-tuning baseline above quantifies this cost rather than assuming it.
Proposed Experiments
The simulator follows the SIMO-OFDM aggregation setup of Chen et al.: estimate-error variance 0.2, receive arrays of 16 to 256 antennas, and channel estimates reused for 1, 5, or 20 rounds. Converters range from 1 to 4 bits, modeled as in Jacobsson et al. In their uplink, 3-bit converters reach about 90% of infinite-resolution spectral efficiency and 1-bit converters about 71%. Device distortion levels are swept over a range of $\kappa^{\mathrm{UE}}$ values. Learning tasks are CIFAR-10 and EMNIST with the Dirichlet 0.1 partitions and ResNet-18 used by ADOTA-FL. Metrics are rounds to target accuracy, final test accuracy, the learning-rate trajectory, the fraction of seeds that diverge, and the measured hypergradient bias. The bias is computed by comparing received inner products against noiseless ones logged in simulation.
Three checks decide the question. First, the full-array bias must grow with channel-estimate reuse and with coarser converters, and must vanish under fresh estimates with high-resolution converters. If it does not, the mechanism is wrong. Second, the split-array bias must stay flat as $N$ grows from 16 to 256 while tracking $\kappa^{\mathrm{UE}}/K$. This confirms that the remaining floor is the device-hardware ceiling and not something the base station can still remove. Third, the range of $\beta$ over which training neither diverges nor freezes must be measurably wider for the split estimator. Sensitivity to $\beta$ is the known weakness of hypergradient methods, and a wider stable range is what would make the method usable without a search over $\beta$ itself.
References
- Online Learning Rate Adaptation with Hypergradient Descent
- Massive MIMO Systems With Non-Ideal Hardware: Energy Efficiency, Estimation, and Capacity Limits
- Over-the-Air Computation in OFDM Systems With Imperfect Channel State Information
- Throughput Analysis of Massive MIMO Uplink With Low-Resolution ADCs
- Adaptive Federated Learning Over the Air
- BOHB: Robust and Efficient Hyperparameter Optimization at Scale