Borrowing Strength Across Sensors to Fix Weather-Driven Bias in Federated Environmental Learning

Picture a network of solar-powered air-quality or solar-farm sensors that jointly train one model to fill gaps in their own readings. None of them ships raw data to a central server. Each round, a sensor trains locally for as many steps as its harvested energy allows. A sensor in full sun might take thirty steps, one under cloud might take two, and one in a storm might take none. This produces two problems at once. First, a naive average pulls the shared model toward the sunny sites. Second, the sites that do little work are often the ones whose data is unusual right now, because the same clouds that starve their batteries also change what they measure. The standard fix divides each sensor's update by its step count. That removes the pull toward sunny sites, but it gives a noisy two-step update the same weight as a careful thirty-step one, and it says nothing about sensors that drop out.

We propose to resolve this using the same property that makes environmental imputation work in the first place: readings across sensors are approximately low-rank, meaning they are driven by a few shared latent factors. If the data is low-rank, the per-sensor update directions in a given round should also lie close to a low-dimensional subspace. The server can then treat the matrix of "sensors × per-step update directions" as a partially and noisily observed low-rank matrix. It weights each sensor's direction by how much work backs it, and it fills in the directions of absent or low-work sensors from the shared structure. These completed directions are aggregated in place of the raw ones. A cheap preliminary diagnostic, the singular-value decay of observed update directions, is pre-registered to predict when this helps and when it should not.

Core Claim

Existing work on uneven local work splits along two lines. FedNova normalizes each client's accumulated update $\Delta_i$ by its step count $\tau_i$ and averages the resulting directions. On non-IID CIFAR-10 it beats FedAvg by up to 9% with local momentum and FedProx by about 10%. Its normalized direction is unbiased but has variance that grows as $\tau_i$ shrinks, and it assumes every scheduled client reports back. The second line comes from energy-harvesting and availability work. Energy-Harvesting Distributed Machine Learning rescales gradients by inverse participation probability, but only for single-step SGD. On Federated Learning with Energy Harvesting Clients adapts the server learning rate to the number of active clients, but assumes equal local steps. CA-Fed reweights by long-run availability under Markovian participation, but ignores step-count variance.

All of these corrections are per-client scalar reweightings, and all assume participation and work are unrelated to the current data. Our claim is narrower and testable. When local work is driven by weather that also shapes the data, and the data is approximately low-rank across sites, low-rank completion of the per-step update directions yields lower imputation error at a fixed energy budget than any scalar reweighting. The gain should be concentrated at the low-energy sites that scalar schemes either drown in noise or forget. The low-rank assumption is the mechanism, sustainable sensing is the regime, and step normalization is the starting point we improve on.

Research Program

Let $\mathbf{d}_{i,t} = \Delta_{i,t}/\tau_{i,t}$ be client $i$'s step-normalized direction in round $t$, observed only for the set $\mathcal{O}_t$ of clients that did at least one step. The server maintains a basis $\mathbf{U}_t \in \mathbb{R}^{p \times r}$ for the update subspace, where $p$ is the number of model parameters and $r$ is a small rank. It also keeps a coefficient vector $\mathbf{v}_i \in \mathbb{R}^r$ per client, and solves

$$ \min_{\mathbf{U}_t,\{\mathbf{v}_{i}\}} \sum_{i \in \mathcal{O}_t} \tau_{i,t}\,\lVert \mathbf{d}_{i,t} - \mathbf{U}_t \mathbf{v}_{i} \rVert^2 + \lambda \sum_{i} \lVert \mathbf{v}_{i} - \mathbf{v}_{i}^{\text{prev}} \rVert^2 . $$

The weight $\tau_{i,t}$ is inverse-variance weighting, because a $\tau$-step average of stochastic gradients has variance roughly proportional to $1/\tau$. The temporal term carries each absent client's coefficients forward from earlier rounds rather than zeroing them. The global step is then $\tau_{\text{eff}} \sum_i p_i \mathbf{U}_t \mathbf{v}_i$, keeping FedNova's effective step size and data weights $p_i$ but summing over all clients, not only those that reported. When $r$ is large enough to fit every observation exactly and $\lambda = 0$, this reduces to FedNova on the observed clients. FedNova is therefore a special case, and the experiments test whether the low-rank restriction helps.

The method draws on two lines of work. The low-rank-completion view under non-random missingness comes from Agarwal et al. and Causal Matrix Completion. Both show that low-rank structure supports recovery even when what is missing depends on the unobserved values, which is exactly the weather-driven case. Their objects are data matrices, whereas ours are update directions inside an optimizer. The local model trained on each sensor is a federated variant of ImputeFormer, a centralized Transformer that builds low-rank priors into attention and uses a Fourier-domain loss. ImputeFormer never considers decentralized training, uneven work, or energy.

Two constraints shape the design. Weighted low-rank approximation with missing entries is NP-hard (Low-Rank Matrix Approximation with Weights or Missing Data Is NP-Hard), so we use a few alternating least-squares sweeps warm-started from the previous round. We make no global-optimality claim. Low-rank methods also degrade sharply when the spectrum is flat (Luo et al.), and low-rank imputers struggle when structural symmetries break (Nie et al.). We therefore expect the method to fail during abrupt local anomalies, and we build the preliminary diagnostic described below around that expectation.

The "preliminary findings" direction enters as methodology rather than decoration. Before any full run, a pilot of a few rounds on a small subset of sensors computes the fraction of update-direction energy captured by the top $r$ singular vectors. We pre-register two predictions: the method beats FedNova only when that fraction exceeds a stated threshold, and its gain grows with the correlation between harvested energy and local data shift. The pilot is cheap. If it fails, that is a decisive negative result that saves the full confirmatory budget. This addresses the under-powered, single-seed pilot reporting criticized in the file (e.g., three runs in FedSum).

Baselines

The naive reference point is FedAvg with energy-limited local steps: each sensor runs whatever steps it can afford and the server averages raw updates. This shows the size of the bias at all. The same rule with a forced minimum common step count shows what we gain by not wasting harvested energy.

The strongest existing method is FedNova combined with inverse-probability participation weighting, which we construct ourselves. FedNova (Tackling the Objective Inconsistency Problem) supplies step normalization. The participation weighting follows the $1/\beta_i$ rescaling of Güler and Yener and the $1/\pi_i$ weighting of CA-Fed, with probabilities estimated online from each sensor's history. It is the strongest competitor because it already corrects both effects that we target, using only scalars. The closest reported numbers are on CIFAR-10, not environmental imputation. FedNova reports up to 9% over FedAvg. Güler and Yener report about 80% accuracy for unbiased scaling, against 64% for energy-agnostic training and 52% for wait-for-all within 1,000 iterations. For federated imputation on PEMS03, Solar, Smart Meters, or Air Quality under energy-driven participation, no reported result was found on Lacuna, so we will produce these baselines ourselves. We will also run CA-Fed alone and the myopic battery-based scheduler with adaptive server learning rate from Shen et al..

Every method shares the same local ImputeFormer-style model, the same energy traces, the same total harvested-energy budget, and the same tuning budget of 20 configurations per method on a held-out validation window. Each runs with at least five pre-registered seeds. Beating the strongest baseline means a lower final test MAE, averaged over seeds with a 95% interval that excludes zero, on at least three of the four environmental benchmarks. The worst-decile-site MAE must improve as well, so the average gain is not bought at cloudy sites.

Three ablations isolate the key ingredient. The first drops the low-rank restriction (full rank, $\lambda = 0$), which recovers FedNova. The second keeps completion but sets all $\tau$-weights equal. The third keeps $\tau$-weighting but carries absent clients forward without the shared basis, using stale directions only. A centralized ImputeFormer trained on pooled data provides an accuracy ceiling, with its data-transfer energy reported alongside.

Why not just…?

Why not just use FedNova with inverse-probability weights? This is the real threat. If participation is unrelated to current data, and low-work directions are not much noisier than high-work ones, it should match our method, and our ablation without the low-rank restriction should do just as well. The experiments settle this by controlling the correlation between energy and data. On the Solar benchmark the correlation is natural, because the node imputing solar output is powered by it. On the other benchmarks it is injected synthetically at graded strengths. If the gap does not grow with that correlation, we concede the scalar correction suffices.

Why not just reuse each absent client's last update? Stale reuse is the third ablation. It ignores the fact that the shared subspace drifts as the global model moves, and it cannot denoise a two-step direction. Under the bursty, sticky availability documented by CA-Fed, stale directions can be many rounds old. We expect stale reuse to lose most at long dropout spells, and we report error as a function of dropout length to check.

Why not just centralize the data and train ImputeFormer in the cloud? For grid-connected stations this is reasonable, which is why it serves as the accuracy ceiling. For battery-limited sensors, transmission dominates the energy budget. Xiang et al. report that transmitting fewer samples extends battery life by over 50%. FedDeep reports 28.32 minutes of training and 0.87 GB of GPU memory federated, against 67.20 minutes for GAT-LSTM and 3.12 GB for the non-federated version. We report joules per unit of MAE improvement for both.

Why not just give every sensor the same small number of steps? This is the minimum-common-steps baseline. It throws away energy the sunny sites have already harvested. Güler and Yener found that the related wait-for-all strategy reached only 52% accuracy in their window.

Proposed Experiments

The testbed uses the four environmental benchmarks evaluated by ImputeFormer: PEMS03, Solar Energy, Smart Meters, and Air Quality. ImputeFormer reports a block-missing MAE improvement of up to 26.5% on PEMS03 and generalization from a 25% to a 95% missing rate on air quality, which anchors that our centralized ceiling is implemented correctly. We add the Jiangsu PM2.5 data from FedDeep if it can be obtained; FedDeep reports a 12-hour forecasting MAE of 12.38. Each sensor or station cluster is one client. Per-round energy comes from solar-generation traces, converted to step and transmission allowances through a fixed joules-per-step and joules-per-byte model. As a sanity check, we reproduce the CIFAR-10 energy-harvesting setting of Güler and Yener to confirm our baselines approach their reported 80%.

The metrics are final and worst-decile-site imputation MAE and RMSE, rounds and joules needed to reach the FedNova-plus-weighting final MAE, and transmitted bytes. The decisive checks are three. First, the pre-registered pilot diagnostic, the top-$r$ energy fraction of update directions, must predict the sign of the gain over FedNova across benchmarks and correlation strengths. Second, the gain must increase with the energy–data correlation and vanish when that correlation is zero. Third, during injected abrupt local anomalies, where the spectrum flattens as Luo et al. predicts, we will show whether adaptively raising $r$ when the diagnostic drops recovers parity with FedNova. If it does not, we report that limitation plainly.