Federated learning trains one model across many devices that never share raw data; each round, devices send model updates to a server, which averages them. Two practical goals pull against each other here. To save bandwidth and on-device memory, we want the shared model to be sparse, meaning most weights are exactly zero, and we want updates to carry only a few coordinates. To protect users, we want the server to see only the sum of updates through secure aggregation, a cryptographic protocol in which devices add random masks that cancel only in the total. The two goals collide because the methods that keep averaged models sparse are nonlinear. They count which devices kept a weight, vote on masks, or threshold coordinates, and they read each device's chosen coordinates in plaintext. Those per-device choices are themselves a privacy side channel. Secure aggregation, for its part, only works when every device sends the same coordinates.
This note proposes a way around the collision. Each round, the server announces which coordinates everyone will send, and it computes that set only from public state: the support of the current sparse global model, plus a small random exploration set drawn from a public seed. Devices send updates on exactly those coordinates under secure aggregation, keep everything else in a local error buffer, and the server applies one sparsity-inducing step to the securely summed result. Two ingredients make this work. The model's own support is a data-independent coordinate set that every device can check, so cryptographic masks cancel. And the one "nonlinear" operation sparse aggregation seems to need, per-coordinate counting of active contributors, is in fact a linear sum that secure aggregation can compute. The expected result is a sparse, bandwidth-cheap training protocol in which the server never sees any single device's mask or update.
Core Claim
Sparse federated training does not need plaintext access to individual updates. If the round's coordinate set is a deterministic function of the published global model and a public seed, then three things can run inside secure aggregation with sparse, fixed-length payloads: dual-space aggregation followed by a server-side proximal step, magnitude-preserving averaging over active contributors, and error-feedback sparsification. The server learns only aggregates. The specific, testable claim has two parts. First, at an equal uplink bit budget, spending most coordinates on the current support should reach higher accuracy at a fixed target sparsity than spending them uniformly at random, which is the only option existing sparse secure aggregation supports. Second, it should do so while removing the per-client mask leakage that current sparsity-preserving methods expose.
Where This Sits
The sparsity-preserving aggregation line motivates the problem and supplies the core method. Federated Composite Optimization names the "curse of primal averaging": averaging sparse local models yields a dense one. Its fix, federated dual averaging, pools accumulated gradients and then applies one global soft-threshold. We reuse that structure because it is linear up to the final step. FedCanon similarly moves the proximal step to the server and adds drift-correcting control variables for heterogeneous data. Neither paper compresses communication or considers what the server sees.
A second group of methods coordinates masks explicitly. Practical Hybrid Gradient Compression (HGC) enforces a shared sparsification mask so the downlink stays sparse. FLASH shows that unstable masks across rounds hurt convergence and fixes the mask from sensitivity profiles or periodic consensus. FedSpaLLM averages each weight only over clients where it is nonzero and expands masks by voting. Complement Sparsification has the server choose which pruned coordinates clients retrain. All four assume the server reads individual masks or coordinates. That is exactly what we remove. The research file flags, without having tested it, that such masks may reveal client class distributions on heterogeneous data.
The privacy line constrains the design. Sparsified Secure Aggregation is the closest prior work: client pairs share seeds so they sparsify the same random coordinates, which lets secure-aggregation masks cancel. It is explicitly limited to data-oblivious random selection, it leaves structure-preserving proximal steps open, and it does not preserve a sparse global model. Our coordinate set is also data-oblivious at the client level, but it is aligned with the model's support rather than uniform, and it feeds a sparsity-producing server step. MineGrad shows that a malicious server can craft broadcast weights so that small adapter updates leak image patches. This is a warning that anything the server broadcasts, including a coordinate set, is an attack surface, and it is why we make the set verifiable. Evaluating Gradient Inversion Attacks and Defenses shows that client-side gradient pruning alone must be extreme to stop inversion. SoK: On Gradient Leakage shows that optimization attacks weaken with multi-step updates and large batches. Both suggest that hiding individual updates behind a sum is a stronger lever than distorting them.
Research Program
Let $w_t$ be the sparse global model at round $t$ and $z_t$ the server's dual accumulator of summed client gradient information. The server publishes $S_t = \mathrm{supp}(w_t) \cup R_t$, where $R_t$ is a set of $m$ random coordinates drawn from a seed produced by commit-reveal among clients, so the server cannot choose it adaptively. Each client $i$ runs local steps, computes its update $\Delta_i$, adds its error buffer, sends only the entries in $S_t$ under secure aggregation, and carries the unsent remainder forward. The server then updates
$$z_{t+1} = z_t + \frac{1}{n_t}\sum_{i} P_{S_t}(\Delta_i + e_i), \qquad w_{t+1} = \mathrm{prox}_{\lambda}(z_{t+1}),$$
where $P_{S_t}$ keeps only coordinates in $S_t$, $e_i$ is client $i$'s error buffer, $n_t$ is the number of surviving clients, and $\mathrm{prox}_\lambda$ is soft- or hard-thresholding at strength $\lambda$. The sum is linear, so standard secure aggregation with dropout recovery applies directly. Payloads have fixed length $|S_t|$, so no padding is needed.
Coordinates outside the support can only become active through $R_t$ and the error buffers. Residual gradient mass for a dormant coordinate builds up locally until the random set samples it, then enters $z$ in one burst. The first analysis question is whether this delayed entry preserves the convergence behavior of dual averaging. We will attempt a bound in which the exploration rate $m/d$, with $d$ the parameter count, enters as a staleness factor, following standard error-feedback arguments. The second design question concerns heterogeneous devices that prune locally inside $S_t$. For that case, clients also secure-sum an indicator vector of their active entries in $S_t$, and the server divides coordinate-wise by these aggregate counts. This reproduces FedSpaLLM-style magnitude-preserving averaging while revealing only counts summed over the cohort, never individual masks. It directly contradicts the file's framing that such counting "requires plaintext inspection," and we will say so and test it.
The third piece is a privacy audit of three levels of server view. The first is the plaintext masks and updates that mask-coordinated methods expose. The second is aggregate-only updates on a support-aligned set. The third is aggregate-only updates on a random set. The audit covers three threats: inferring a client's label distribution, running optimization-based inversion on the aggregate as cohort size varies, and an active server that tries to steer the coordinate set toward coordinates that isolate a target client's features, as in Gap 4. Clients verify that the announced set equals the support of the broadcast model plus the seeded random set, which bounds the mask-steering threat. We state plainly that this does not stop a server from poisoning the broadcast weights themselves, which is the attack MineGrad uses. That case gets its own reported evaluation rather than a claim of defense.
Baselines
The naive reference point is dense federated averaging, with no compression and no secure aggregation. This gives the accuracy ceiling and the bandwidth floor. On FEMNIST, Complement Sparsification reports dense training at 76.3% accuracy. On CIFAR-10 with ResNet-18, HGC reports uncompressed averaging at 90.52%.
The strongest competitor on the precise question, secure aggregation with sparse payloads, is Sparsified Secure Aggregation. Here clients sparsify the same random coordinates and securely sum them. Its reported gain is a 7.8× bandwidth reduction over standard secure aggregation on CIFAR-10 and 17.9× on MNIST. We will run it both as published and composed with FedCanon's server-side proximal step, so that it also produces a sparse model. This composition, with the same budget $|S_t|$ spent entirely on random coordinates, is the decisive ablation. It isolates our key ingredient, support-aligned coordinate selection. Beating it means higher final accuracy at the same target density, for example 10% and 5% nonzero weights, and the same total uplink bits including cryptographic overhead. Otherwise it means reaching the same accuracy with materially fewer bits.
The strongest plaintext sparse methods show the accuracy we would give up by hiding individual updates. For HGC, the file reports 90.26% accuracy on CIFAR-10 ResNet-18 with 15,286× uplink and 2,911× downlink compression. For FLASH, the file reports about 10.1% higher accuracy than ZeroFL with about 10.26× fewer bits. For Complement Sparsification, it reports 72.5% on FEMNIST with 93.2% less uplink traffic. We do not expect to match HGC's compression ratio, because predictive coding of per-client residuals is incompatible with secure summation. Success against these methods is defined as staying within a stated margin of their accuracy at equal density while eliminating per-client exposure. FedCanon and federated dual averaging run dense serve as the uncompressed composite references; no reported result found on Lacuna for either method's CIFAR-10 accuracy.
For privacy, the reference defense is gradient pruning at 0.9 with InstaHide. Evaluating Gradient Inversion Attacks and Defenses reports LPIPS 0.73 versus 0.45 undefended at batch 32, with accuracy falling from 93.37% to 86.10%. All methods share data partitions, client sampling, local epochs, and a fixed hyperparameter-search budget per method, with thresholds tuned to hit the same final density.
Why not just…?
Why not just combine FedCanon with random-coordinate secure aggregation? This is the real threat, because both pieces exist and their composition is SecAgg-compatible. The argument against it is that under a 5–10% density target, most random coordinates land on weights the proximal step will zero anyway, so the budget is spent mostly on noise. That is an argument, not evidence. The equal-bit ablation above settles it. If uniform selection matches ours, the contribution shrinks to the counting trick and the privacy audit, and we will report that.
Why not just add differential privacy noise? Noise distorts every update, and the file's evidence suggests that this costs accuracy. Mixed Quantization Enabled FL reports that differential privacy baselines fall to roughly 78–85% on MNIST versus 96.67% for their deterministic defense. Differential privacy is also orthogonal: it can be added on top of a secure sum at lower noise per client. We will include a variant with central noise added to the aggregate, rather than treat differential privacy as a rival.
Why not just keep HGC or FLASH shared masks in plaintext? They are more compressive. However, their masks are derived from, or exposed alongside, individual client signals, and the server reads individual updates. Our label-inference experiment will measure how much a server learns about each client's class mix from those views. That number is the cost this proposal eliminates.
Why not just use a static FLASH-style mask under secure aggregation? A fixed mask is also public and SecAgg-compatible. But it can never recover coordinates pruned early, and FLASH itself shows that mask choice matters. Setting the exploration size $m=0$ with a frozen support is exactly this alternative, and it runs as an ablation.
Proposed Experiments
The main benchmarks are FEMNIST and Twitter from LEAF, which have natural per-writer and per-user heterogeneity, and CIFAR-10 with ResNet-18 under Dirichlet partitions at two heterogeneity levels. The active-attack study uses CIFAR-100 with a ViT and adapters, matching MineGrad's setting, where PEFTLeak reports 85.9% patch recovery at batch 32. We will report five kinds of metrics. The first is accuracy at target densities of 10% and 5%. The second is cumulative uplink and downlink bits including cryptographic overhead. The third is simulated wall-clock time, with the caveat that coordinate sparsity may not translate to on-device speed without sparse kernels. The fourth is label-distribution inference error from each server view. The fifth is inversion quality, as LPIPS and PSNR, against aggregate size.
Decisive Checks
Three results decide whether the idea holds. First, at equal bits and density on CIFAR-10 and FEMNIST, support-aligned selection must beat random selection with the same server proximal step; if it does not, the core mechanism fails. Second, securely counted magnitude-preserving averaging must match FedSpaLLM-style plaintext counting in accuracy when clients prune locally, which would show that the claimed cryptographic clash is avoidable. Third, the privacy audit must show that label inference from our server view falls to near the level of a random-mask view, while plaintext mask methods leak measurably. It must also measure inversion on aggregates over cohorts of 2 to 32 clients, which tells practitioners what minimum cohort size the protocol needs. A negative outcome on the active-weight attack will be reported as the limit of the method, not hidden.
References
- Federated Composite Optimization
- FedCanon: Non-Convex Composite Federated Learning With Efficient Proximal Operation on Heterogeneous Data
- Practical Hybrid Gradient Compression for Federated Learning Systems
- Revisiting Sparsity Hunting in Federated Learning: Why does Sparsity Consensus Matter?
- FedSpaLLM: Federated Pruning of Large Language Models
- Complement Sparsification: Low-Overhead Model Pruning for Federated Learning
- Sparsified Secure Aggregation for Privacy-Preserving Federated Learning
- MineGrad: Gradient Inversion Attacks on LoRA Fine-Tuning
- Evaluating Gradient Inversion Attacks and Defenses in Federated Learning
- SoK: On Gradient Leakage in Federated Learning
- Mixed Quantization Enabled Federated Learning to Tackle Gradient Inversion Attacks
- LEAF: A Benchmark for Federated Settings