Federated learning trains a shared model by having many devices send model updates, not raw data, to a server. Two separate pressures act on those updates. Bandwidth is scarce, so updates are compressed by dropping coordinates, quantizing them to a few bits, or restricting them to a small low-rank subspace. Updates also leak, because an attacker who sees a gradient can often reconstruct the images or sentences that produced it. The standard defense against leakage is differential privacy (DP), which clips each update and adds calibrated noise so that any one person's data changes the output distribution only slightly. A single number, epsilon, summarizes that guarantee. Recent work shows that compression and DP interact in useful ways. Compression can filter out DP noise, it can lower the noise that is needed, and quantization can stand in for the noise entirely. These papers compare methods at what they call the same epsilon. The epsilons, however, come from accountants of very different tightness, and nobody checks whether two mechanisms with equal epsilon actually leak equally to a realistic reconstruction attacker.
This note proposes to build an iso-epsilon, iso-bandwidth testbed. Every compressed private mechanism will be accounted with tight numerical privacy-loss accounting, including a new exact accountant for randomized quantization, and then run against strong prior-guided gradient inversion attacks. The central question is whether the placement of compression relative to the noise changes how much an attacker recovers, once both the formal guarantee and the uplink bits are held fixed. Compression can come after the noise, or before it, which concentrates the privacy budget on fewer coordinates. It can also replace the noise. Either answer is useful. If leakage differs at matched epsilon, epsilon alone is an insufficient design target for compressed federated learning. If leakage does not differ, practitioners can safely choose purely on utility and bits.
Core Claim
The hypothesis is this. At matched tight epsilon and matched bits per round, mechanisms that compress before adding noise will give the best accuracy. These are random sparsification with a reduced clipping norm (Wei et al.), training only a small "bridge" matrix inside frozen low-rank adapters (Fed-SB), and randomized quantization that serves as its own noise (Youn et al.). The same mechanisms will also give measurably higher average-case reconstruction quality than mechanisms that compress after noising, such as top-k or low-rank compression of an already-noised gradient (Lin).
The reasoning runs as follows. DP bounds the worst case over neighboring datasets, and every mechanism here is calibrated to that worst case. A pre-noise compressor spends the same budget on fewer, cleaner coordinates. Attacks with generative priors are known to succeed from very few coordinates: GIAS reconstructs images "even when 99% of gradient values are removed" (Jeon et al.). Post-noise compression, by contrast, is post-processing of a noisy vector and cannot add information for any attacker. The claim is falsifiable. If reconstruction scores at matched epsilon fall within seed-to-seed variation across all placements, the hypothesis is wrong.
The first new element is the measurement discipline: tight accounting is used as a ruler so that cross-mechanism comparisons are actually at equal privacy. The second is the empirical answer about compression placement. Neither appears in the closest work. Lin, Wei et al., Youn et al. and Fed-SB all report utility against an epsilon, but none runs a reconstruction attack. Sultana et al. run attacks but have no formal epsilon at all.
Background and Where It Falls Short
On the accounting side, the Moments Accountant (Abadi et al.) and its Rényi-divergence descendants track a few moments of the privacy loss and give closed-form but loose bounds. The privacy loss is the log-ratio of output probabilities under two neighboring datasets. The Fourier Accountant (Koskela et al.) instead composes the full privacy-loss distribution by FFT convolution, and "Connect the Dots" (Doroshenko et al.) gives provably optimal pessimistic discretizations of that distribution. Both are built for continuous one-dimensional mechanisms such as the Gaussian. Koskela et al. explicitly leave discrete mechanisms open. This matters directly here, because randomized quantization is discrete. Youn et al. analyze their mechanism only with scalar Rényi bounds, and OPTM only per scalar operation. These accounting papers supply the method this proposal reuses and extends.
On the leakage side, a survey of gradient inversion (Zhang et al.) describes optimization attacks that match the direction of a dummy gradient to the observed one. Cosine matching beats Euclidean matching in these attacks. The survey also notes the absence of computable metrics linking DP parameters to inversion risk; that finding motivates this proposal. GIAS (Jeon et al.) searches in a GAN's latent space and defeats both large batches and heavy sparsification. LAMP (Dimitrov et al.) recovers text from BERT gradients using a language-model prior. Wang et al. show that a passive server tracking a client's updates over many rounds can reconstruct features. These papers supply the attacks.
On the compression-meets-DP side, the papers make different claims. Lin shows that compressing a noised gradient lowers total error, and his Denoise method adds temporal averaging. Wei et al. show that random sparsification to a fraction $s$ bounds sensitivity by $\sqrt{s}C$, so the noise scale shrinks. Fed-SB avoids LoRA's multiplicative noise blow-up by sending only an $r \times r$ matrix between fixed public adapters. Sultana et al. defeat DLG with mixed-precision quantization whose scales stay secret on the device. The Sultana result is a partial counterpoint to this note's thesis, since it suggests heuristic compression alone can stop attacks. Its security rests on secrecy of metadata, though, and it has never been tested against an attacker who estimates the scales or uses a generative prior. This proposal runs exactly those tests.
Research Program
The first component is a tight accountant for vector randomized quantization. The scalar randomized quantization mechanism has finite output support. Its privacy-loss distribution can therefore be computed exactly as a probability mass function rather than discretized, and then composed across rounds and subsampling with the same FFT machinery as Koskela et al.. The harder part is a vector of coordinates under an overall norm bound on the update: the worst-case pair of neighboring updates is not obvious. This will be handled by computing a pessimistic bound. The bound is obtained by searching over how the sensitivity is spread across coordinates, using convexity of the hockey-stick divergence to reduce the search, and it is reported alongside the Rényi-based bound of Youn et al.. The size of that gap is itself a result. If the gap is small, Rényi accounting was adequate for this mechanism.
The second component is iso-epsilon calibration. Every mechanism's noise or quantization parameters are tuned so that its tightly accounted epsilon after the full training run equals a target value, at 1, 3 and 8 with delta of $10^{-5}$, to within 0.05. The uplink bits per round are set equal by adjusting sparsity, rank or bit-width. All mechanisms share Poisson client subsampling so that amplification is accounted identically.
The third component is a fixed attack suite applied to every mechanism. It contains a cosine-matching optimization attack, GIAS with a pretrained image generator, and LAMP for text, each run both in a single-round setting and in a multi-round tracking setting following Wang et al.. Each attack receives full knowledge of public structure. That includes the frozen adapters in Fed-SB and the sparsity mask distribution in DP-SparFL. For the Sultana defense, the attack suite adds a variant that estimates the quantization scales from the transmitted integers.
Baselines
The naive reference point is uncompressed federated averaging with no DP. This upper-bounds utility and shows how bad leakage is without defense. Sultana et al. report near-perfect DLG reconstruction at 55.3 dB PSNR in this regime (Sultana et al.). The second reference is uncompressed Gaussian DP-SGD at matched epsilon, accounted with Connect-the-Dots. It is run on the same data partitions and model, with the same rounds and the same hyperparameter search budget of a fixed number of trials over learning rate and clipping norm per method.
The strongest existing compressed-private competitor for language fine-tuning is Fed-SB. It reports 70.33% accuracy on SNLI at $\epsilon=1$, against 49.57% for standard federated LoRA. Fed-SB is the strongest competitor because it already wins on utility and bits simultaneously. It will be run with its original adapter initialization from 0.1% server data. Its epsilon is recomputed with tight accounting so that the noise can be lowered if the original accountant was loose. Beating Fed-SB on this question does not mean higher accuracy. It means showing that a post-noise mechanism at the same epsilon and bits recovers within 2 points of Fed-SB's accuracy while yielding lower LAMP ROUGE-2. Alternatively, it means establishing that Fed-SB's leakage is no worse, which would make Fed-SB the recommended design.
For images, the strongest competitors are DP-SparFL (Wei et al.) and Denoise with PowerSGD (Lin). No reported CIFAR-10 accuracy for either was found on Lacuna. Both will therefore be rerun under the shared protocol rather than compared to published figures. Randomized quantization (Youn et al.) is run on EMNIST, where no reported numerical result was found on Lacuna. Sultana's mixed quantization is included as a no-formal-guarantee reference, with its reported 96.67% MNIST accuracy as the utility target.
The ablation that isolates the key ingredient keeps the compressor fixed and moves it. Random sparsification at fraction $s$ is placed either before the noise, with the reduced clip, or after the noise, with the full clip. Both versions are calibrated to the same epsilon and bits. The second ablation isolates tight accounting: all mechanisms are recalibrated to equal Rényi-derived epsilon, and the analysis checks whether the leakage and utility rankings change.
Why not just…?
Why not just keep using Rényi accounting, since it is simpler? For Gaussian mechanisms the gap to tight accounting may be modest. For a discrete quantizer it is unknown, and Koskela et al. show that moment-based bounds overestimate epsilon. If the looseness differs across mechanisms, "equal epsilon" comparisons are biased toward whichever method has the looser bound, because that method is effectively run with less privacy. The Rényi-matched ablation settles this directly. If rankings are unchanged, the tight accountant is unnecessary for this question, and the note will report that.
Why not just compare mechanisms at the same noise multiplier? Randomized quantization has no additive noise and Fed-SB's noise lives in a different dimension, so a shared noise level does not exist. Epsilon is the only common currency, which is exactly why it must be computed tightly.
Why not just run empirical attacks, as Sultana et al. did, and skip formal accounting? Attack-only evaluation shows that a specific attack fails. It cannot show that a stronger one would also fail. The scale-estimation attack and GIAS in this testbed test exactly this weakness of secret-metadata defenses. A collapse of Sultana's 8.1 dB PSNR under those attacks would be direct evidence that formal accounting is needed.
Why not just use more local steps or larger batches? Zhang et al. report only a 60% drop in recovery from 10 local iterations. GIAS stays effective at batch size 32 (Jeon et al.), and Chu et al. extract text across thousands of aggregated users. These knobs are kept fixed across mechanisms rather than treated as alternatives.
Proposed Experiments
The image track trains a small CNN and ResNet-18 on CIFAR-10 and Fashion-MNIST with IID and label-skewed client splits. Utility is reported as test accuracy. Leakage is reported as PSNR and SSIM from the cosine attack and GIAS, along with label-recovery rate. GIAS's reported 15.58 dB against 13.78 dB for pixel attacks on undefended gradients (Jeon et al.) serves as the sanity check that the attack implementation is faithful.
The text track fine-tunes BERT on SNLI with Fed-SB, federated LoRA, and post-noise compression. Utility is measured by accuracy, and leakage by LAMP's ROUGE-2 on CoLA and SST-2 batches. The randomized-quantization track runs on EMNIST and reports the tight-versus-Rényi epsilon gap. All tracks report bits per round and accounting runtime.
The decisive check is a comparison of matched-epsilon, matched-bits pairs that differ only in compression placement. The thesis holds if the gap in reconstruction quality between such pairs exceeds 2 dB PSNR, or 5 ROUGE-2 points, beyond the spread across five seeds, consistently across the three epsilon targets. It fails if the gaps vanish. A secondary check asks whether multi-round tracking widens the gap. A wider gap under tracking would indicate that per-round clean coordinates add up across rounds faster than the composed epsilon suggests to practitioners. If confirmed, either result yields a concrete design rule for choosing among compression schemes at a fixed privacy budget.
References
- Gradient sparsification for efficient wireless federated learning with differential privacy
- Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
- Randomized Quantization is All You Need for Differential Privacy in Federated Learning
- On the Interaction Between Differential Privacy and Gradient Compression in Deep Learning
- Gradient Inversion with Generative Image Prior
- Mixed Quantization Enabled Federated Learning to Tackle Gradient Inversion Attacks
- Deep Learning with Differential Privacy
- Computing Exact Guarantees for Differential Privacy.
- Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributions
- Privacy-Aware Randomized Quantization via Linear Programming
- A Survey on Gradient Inversion: Attacks, Defenses and Future Directions
- LAMP: Extracting Text from Gradients with Language Model Priors
- Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
- Panning for Gold in Federated Learning: Targeted Text Extraction under Arbitrarily Large-Scale Aggregation