In federated learning, many phones or edge boards train a shared model by sending gradient updates instead of raw data. Two separate communities decide what goes into those updates. Systems researchers compress them, usually by keeping only the largest-magnitude entries (Top-$k$ sparsification), so that slow clients can finish uploading on time. Privacy researchers show that the same updates can be inverted back into the training images. Nobody has asked whether the entries Top-$k$ keeps are the ones that give the data away. There is reason to think they are. The attacks that work best match gradient direction, and the largest coordinates dominate that direction (Inverting Gradients). The closed-form attack on fully connected layers reads the input directly off the largest weight and bias gradient rows. So a bandwidth-optimal compressor may concentrate the upload on the most revealing part of the gradient.
We propose to treat the selection of transmitted gradient blocks as one allocation problem with two budgets. A client decides, per round, which blocks of its gradient to upload. It maximizes their value for training subject to its uplink budget and a cap on a fast, closed-form estimate of how well an attacker could reconstruct its data. The allocator itself is a cheap greedy knapsack-style procedure of the kind that runs in seconds in systems work. The privacy estimate comes from inversion influence functions, which predict reconstruction error from a gradient perturbation without running the attack. If this works, compression stops being privacy-neutral and becomes the defense, at no extra bandwidth.
Core Claim
At an equal number of uploaded bytes and equal wall-clock training time, leakage-aware block selection should drive gradient-inversion reconstructions to the "unrecognizable" range while losing little accuracy relative to Top-$k$. It should also do better than three alternatives: stacking an existing privacy defense on top of Top-$k$, sending random blocks, or quantizing everything uniformly. The new ingredient is the selection rule, not a new defense primitive.
Research Program
The first piece of prior work supplies the systems side. FedCG (Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression) jointly picks clients with a submodular facility-location objective. It then assigns each client a Top-$k$ compression ratio by linear programming, so that all selected clients finish uploading at the same moment. It never measures inversion risk. We keep its per-client byte budget $b_n$ and its error-feedback memory, and we replace the rule for which coordinates fill that budget.
The second piece supplies the privacy estimate. The inversion influence function (Understanding Deep Gradient Leakage via Inversion Influence Functions) applies the implicit function theorem to the gradient-matching attack. It gives the attacker's reconstruction error as a linear function of the perturbation applied to the true gradient. It is computed with Jacobian-vector products, and the paper reports it is over 500× faster than running the attack on ResNet-152. The paper also shows leakage is highly non-uniform across directions and leaves open how to use this for designing updates. We use it for exactly that. Dropping a set of blocks is a perturbation $\delta = -g_{\bar S}$, where $g$ is the client gradient and $\bar S$ the unsent blocks. The predicted reconstruction error is $\|M\,g_{\bar S}\|$, where $M$ is the influence operator, applied through Jacobian-vector products rather than formed explicitly.
The per-round client problem is then
$$\max_{S} \; \sum_{j \in S} \|g_j\|^2 \quad \text{s.t.} \quad \sum_{j \in S} c_j \le b_n, \qquad \|M\, g_{\bar S}\|^2 \ge \tau .$$
Here $S$ is the set of transmitted blocks, $g_j$ is block $j$ of the gradient, and $c_j$ is its size in bytes. The objective is retained gradient energy, which is the quantity Top-$k$ implicitly maximizes. The threshold $\tau$ is a minimum predicted reconstruction error, calibrated once per architecture against real attacks so that it corresponds to a target PSNR.
We solve this with a Lagrangian greedy rule. We rank blocks by energy minus $\mu$ times their marginal contribution to leakage, divided by bytes. We then fill the budget and bisect on $\mu$ until the leakage constraint holds. This follows the rank-then-place pattern that gets within small gaps of exact solvers in placement problems (Online Deployment Algorithms for Microservice Systems with Complex Dependencies, Probabilistically Survivable MASs). The leakage term is a quadratic in the unsent blocks, so marginal contributions interact. We recompute them after each batch of additions, which is a local-search step in the style of Resource Sharing through Multi-Round Matchings. On small instances we compare the result against an exact mixed-integer solver to measure the optimality gap.
We work at block granularity (output channels, attention heads, and fully connected rows including bias) rather than per coordinate. This keeps the number of Jacobian-vector products on a Jetson-class client to tens per round. It also means the rule can explicitly withhold the few fully connected rows that make the analytic attack exact. Soteria (Soteria) does something similar but only by pruning the penultimate representation, and it ignores bandwidth. Two consequences need handling. Withheld blocks accumulate in error feedback and are eventually sent, so we charge leakage on the accumulated vector at the round it is sent. Clients whose leakage constraint cannot be met within budget are reported back to FedCG-style client selection as higher-cost participants.
Three risks are specific to this design. The first is that the influence function is a linearization, and dropping 90–99% of blocks is not a small perturbation. Whether its ranking of blocks remains correct in that regime is the first thing we test. The second is that the SoK reality check (SoK: Gradient Leakage in Federated Learning) finds that several local steps already make reconstructions unrecognizable (LPIPS > 0.1). This constrains where the method matters. We target single-step or few-step updates, small batches, and fine-tuning of pretrained models, where Inverting Gradients still recovers recognizable images from batches of 100. The third is a limit of scope: we defend only against a passive, honest-but-curious server, not the active weight-tampering attacks of OASIS.
Baselines
The naive reference is uncompressed FedAvg with no defense. It fixes the attack ceiling on our setup. For context, Mixed Quantization Enabled Federated Learning reports DLG PSNR of 55.3 on CIFAR-10 with LeNet-5 and no defense. Concealing Sensitive Samples reports 59.20 on MNIST.
The strongest systems competitor is FedCG with Top-$k$ and error feedback. It already solves the bandwidth side and is exactly what our method replaces. On its Jetson testbed it reports a 4.9× convergence speedup for AlexNet on CIFAR-10, and a 55.4% training-time reduction versus FlexCom/AdaSample. We rerun it with identical per-client byte budgets, bandwidth traces, client selection and learning-rate tuning, and swap only the selection rule. Beating it means matching its time-to-target-accuracy within 10%. Matching accuracy alone is not enough, because we also need attack PSNR below roughly 12 dB, LPIPS above the SoK's 0.1 threshold, and SSIM near the 0.15 that DCS² reaches.
The strongest privacy competitors are Soteria stacked on FedCG's Top-$k$, and DCS² stacked on the same. Soteria reports up to 160× higher reconstruction error at no accuracy loss. Concealing Sensitive Samples reports TinyImageNet ResNet-18 accuracy of 52.06% for Soteria and 65.68% for DCS², with CIFAR-10 SSIM 0.15. LaDP-FL (Local Layer-wise Differential Privacy) is a layer-wise noise allocator. It is added as a baseline because it is the nearest existing "allocation" defense, though it uses KL divergence and weight-norm heuristics and has no bandwidth model. All stacked baselines get the same byte budget and are tuned with the same number of trials. We also charge their client compute time to the wall clock, since DCS²'s decoy synthesis runs an inner optimization on every round.
Two ablations isolate the selection rule. Random-block selection at equal bytes tests whether any non-magnitude selection already confuses the attacker. Leakage-only selection sets $\mu$ to infinity and tests whether the energy term matters.
Why not just…?
Why not just quantize everything to 2 bits? This is the most serious threat. The SoK (SoK: Gradient Leakage in Federated Learning) reports that uniform 2-bit quantization blocks current attacks with less than a 5% accuracy drop. Our answer is not that it fails, but that it is inflexible. It imposes one rate on clients with very different bandwidth, which FedCG shows creates stragglers. It may also not survive attacks with a strong image prior. We will run 2-bit quantization, and Top-$k$ at 2 bits, at matched bytes against both the signed-Adam cosine attack and the GAN-prior attack of Gradient Inversion with Generative Image Prior. Suppose 2-bit quantization matches our leakage and accuracy under heterogeneous budgets. Then the contribution shrinks to a negative result about Top-$k$, and we will report it as such.
Why not just send random blocks instead of the largest? Random selection is free and destroys the magnitude structure that attackers exploit. However, with error feedback it converges more slowly than Top-$k$, and it keeps fully connected bias rows with some probability every round, so the exact analytic attack still fires on those rounds. The random-block ablation measures both effects directly. If it matches us, the influence function is unnecessary.
Why not just stack Soteria on Top-$k$? Composition is simpler and each piece is published. But Soteria only touches the penultimate layer, and Top-$k$ then spends the byte budget on whatever remains most salient in earlier layers. This is the regime where the Geiping analytic result says depth offers no protection. The stacked baseline above tests this. We expect the gap to appear on ResNet-18 and ViT-small rather than LeNet.
Why not just run more local steps? For most of our targets this would work, and we say so. We therefore restrict claims to the single-step, small-batch and fine-tuning regimes where Inverting Gradients shows leakage persists. We report the $H$ at which the no-defense baseline already becomes safe, so readers see where the method is irrelevant.
Proposed Experiments
The decisive check comes first and is cheap. On CIFAR-10 with ResNet-18, we take single-batch gradients at initialization and after partial training. We vary the kept fraction from 50% down to 1%, under Top-$k$, random, and influence-ranked selection. We then correlate the influence function's predicted reconstruction error with the actual PSNR and LPIPS of the cosine attack. If the rank correlation across selections falls below about 0.6 at high sparsity, the linearization fails there. In that case we fall back to calibrating a per-layer leakage table with real attacks offline, and drop the online influence computation.
The same experiment directly answers whether Top-$k$ is more revealing than random selection at equal bytes. That result would be publishable on its own.
The main study uses 100 simulated non-IID clients with latent-Dirichlet partitions, as in FedCG. It covers CIFAR-10, CIFAR-100 and TinyImageNet, with CelebA and LFW for identity-level reconstructions and FEMNIST for writer-partitioned data. Models are ResNet-9/18 and a small ViT. Per-client uplink budgets come from FedCG's LP under heterogeneous bandwidth traces. Attacks are DLG, the cosine signed-Adam attack, the GAN-prior attack, and the analytic fully connected attack, all run by an honest-but-curious server on every transmitted update, including error-feedback flushes. We report accuracy, wall-clock time to target accuracy, uploaded bytes, and the PSNR/SSIM/LPIPS distribution over clients, with emphasis on the worst-case client, since leakage is non-uniform.
We also measure the allocator's per-round runtime on Jetson-class hardware against FedCG's reported 1.2-second optimization loop, and its optimality gap against an exact solver on small instances. If the budget allows, we repeat the headline comparison on a physical Jetson testbed.
References
- Inverting Gradients - How easy is it to break privacy in federated learning?
- Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression
- Understanding Deep Gradient Leakage via Inversion Influence Functions
- Online Deployment Algorithms for Microservice Systems With Complex Dependencies
- Probabilistically Survivable MASs
- Resource Sharing through Multi-Round Matchings
- Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective
- SoK: Gradient Leakage in Federated Learning
- OASIS: Offsetting Active Reconstruction Attacks in Federated Learning
- Mixed Quantization Enabled Federated Learning to Tackle Gradient Inversion Attacks
- Concealing Sensitive Samples against Gradient Leakage in Federated Learning
- Local Layer-wise Differential Privacy in Federated Learning
- Gradient Inversion with Generative Image Prior