Federated learning trains one model across many devices or hospitals without moving their data: each client trains locally and sends only a model update to a server. Three problems collide when such a system must also decide which examples to send for human labeling, which is active learning. On imbalanced data, a good acquisition rule deliberately picks the rare-class examples the model is missing. Those few examples are the ones whose updates are most easily inverted back into images by a curious server. They are also the ones whose small, diffuse update signal is most likely to be thrown away when clients compress updates to save bandwidth. The examples the system works hardest to acquire are therefore both the least protected and the most likely to be lost in transit.
This note proposes to measure that bind directly and to break it with one mechanism. Clients already compress their updates, so we make the compressor decide which coordinates to keep using two scores. The first is how much each coordinate matters to the model's outputs on a class-balanced calibration set taken from the actively labeled pool. The second is how much each coordinate is dominated by a single training example, which is a proxy for how easily that example can be reconstructed from it. The bandwidth budget then becomes a privacy lever as well: we spend the kept coordinates on directions that carry shared, rare-class-relevant signal, and we drop directions that are fingerprints of individual samples.
Core Claim
We make two claims. The first is empirical: in federated active learning under class imbalance, an example's acquisition value correlates positively with how well it can be reconstructed from the client's update, and its contribution survives standard compression worse than majority-class contributions do. The second is methodological: compression masks chosen by balanced output sensitivity minus per-coordinate sample dominance give better rare-class accuracy and lower reconstruction quality at the same upload size than magnitude Top-$k$, discrepancy-aware calibration with a random calibration set, or median binarization. The second claim must be held to account against a constraint from prior work. Gradient Inversion with Generative Image Prior, which inverts gradients by searching a generative model's latent space, reports reconstruction that stays robust even when 99% of gradient values are pruned. Our hypothesis is that this robustness holds for magnitude-based or random pruning and not for pruning that specifically removes sample-dominated directions. If it also holds for our masks, the privacy half of the proposal fails, and we will report that.
Research Program
The first stage is diagnostic. We build a federated active learning testbed with Dirichlet label skew across clients and a global long-tail imbalance ratio, following the protocol of Re-Thinking Federated Active Learning Based on Inter-Class Diversity. That paper's method, LoGo, clusters unlabeled points in a "hallucinated" last-layer gradient space using a local model and then picks the most globally uncertain point in each cluster. It never considers what the subsequent uploads reveal or what compression does to them. Using LoGo as the acquisition rule, we log for every acquired example three quantities: its class frequency; the reconstruction quality an attacker reaches from the round's update (generative-prior inversion and plain pixel-space gradient matching); and the fraction of its per-sample gradient energy that survives each compressor. The diagnostic succeeds if rare-class acquisitions are measurably more reconstructable and less preserved than majority-class ones.
The motivation for expecting this comes from two papers. Concealing Sensitive Samples against Gradient Leakage in Federated Learning shows that inversion succeeds when per-sample gradients are poorly entangled, meaning they point in nearly separate directions. A lone rare-class example in a batch is the extreme case of this. Enhancing Communication Compression via Discrepancy-aware Calibration for Federated Learning shows that update magnitude is a poor guide to functional importance. Its calibration set of 32–64 random samples will usually contain no minority examples at a 1% minority ratio, such as the ECG5000 regime studied in Active Learning with Fast Model Updates and Class-Balanced Selection for Imbalanced Datasets.
The second stage is the method. Let $\Delta$ be a client's round update with coordinates indexed by $j$. Let $u_j$ be the change in model outputs, measured on a calibration set drawn class-stratified from the client's actively labeled pool, when coordinate $j$ is dropped. This is the discrepancy-aware score with the calibration sampling changed. Let $g_{i,j}$ be the per-sample gradient of labeled example $i$ at the start of the round, and define the dominance $d_j = \max_i |g_{i,j}| / \sum_i |g_{i,j}|$, which is near 1 when a single example owns that direction. The client sends the coordinates in the mask
$$M^\star = \arg\max_{|M| \le k} \sum_{j \in M} \left( u_j - \lambda\, d_j \right),$$
where $k$ is the upload budget and $\lambda \ge 0$ trades utility against exposure. Error feedback (accumulating the dropped residual for later rounds) is turned off for high-dominance coordinates, so that sample fingerprints are not delayed and re-sent. Per-sample gradients are computed only over the labeled examples in the current local batch and, where cost demands, at parameter-block granularity rather than per scalar. We will report client FLOPs and memory because this method is meant to be cheaper on-device than generating decoy images.
The third stage connects back to acquisition, and only lightly. The class-stratified calibration set is exactly the balanced labeled pool that a class-aware acquisition rule produces. In that sense the active learner supplies the compressor's notion of importance. We do not change LoGo's selection rule, so that any gain can be attributed to the update path.
Baselines
The naive reference is federated active learning with LoGo acquisition, uncompressed FedAvg uploads, and no defense. It fixes the ceiling for rare-class accuracy and the floor for privacy. Next is the same pipeline with magnitude Top-$k$ at matched upload size, the default compressor most systems would deploy.
The strongest compression competitor is discrepancy-aware calibration as published in Enhancing Communication Compression via Discrepancy-aware Calibration for Federated Learning. It ranks parameters by their true output shift on a small random calibration set. On CIFAR-10 with ViT-tiny at a 0.01 compression ratio it reports 29.62% accuracy against 21.03% for Top-$k$. This is the strongest competitor because it already rejects magnitude as a proxy for importance, so beating it isolates the value of balanced calibration and the dominance penalty. We run it with the same calibration-set size, the same $k$, and the same local steps. We give it the same hyperparameter search budget over its learning rate and calibration size as our method gets over $\lambda$.
The strongest combined privacy-and-communication competitor is EdgeDetect: Importance-Aware Gradient Compression with Homomorphic Aggregation for Federated Intrusion Detection. It binarizes gradients around their median magnitude, encrypts them with Paillier homomorphic encryption, and handles imbalance by local oversampling. On CIC-IDS2017 it reports 98.0% accuracy, 97.9% macro F1, a 450 MB to 14 MB upload reduction, and inversion PSNR reduced from 31.7 dB to 15.1 dB. We reproduce it on CIC-IDS2017 inside the active learning loop, with the same labeling budgets. We add a variant that keeps only its binarization, so that compression and encryption effects can be separated.
The privacy-defense competitor is DCS² from Concealing Sensitive Samples against Gradient Leakage in Federated Learning. It synthesizes decoy samples whose gradients mimic sensitive ones and projects the result back onto the batch subspace, followed by the same compressor. It reports MNIST reconstruction PSNR falling from 59.20 dB to 7.84 dB, and TinyImageNet accuracy of 65.68% against Soteria's 52.06%. Two-sided DP-SGD with clipping and Gaussian noise is also a baseline, tuned to match our method's reconstruction PSNR.
Two ablations isolate the key ingredient. In the first, $\lambda = 0$ with stratified calibration, which shows what balanced calibration alone buys. In the second, $\lambda > 0$ with random calibration, which shows what the dominance term alone buys. We count the method as beating the field only if it Pareto-dominates discrepancy-aware calibration and EdgeDetect-binarization at equal bytes. That means higher minority-class recall and macro F1, together with lower attacker PSNR or SSIM, across at least two of the three datasets. For LoGo itself, the file reports only a pairwise win rate over BADGE (12.0 wins per 2.4 losses). There is no reported result on Lacuna for LoGo's absolute accuracy on MedMNIST or CIFAR-10, so we will establish that number ourselves.
Why not just…?
Why not just use secure aggregation or homomorphic encryption, as EdgeDetect does? Encryption hides individual client updates, but the server still sees the sum. When a rare class lives on one or two clients, which is common under Dirichlet skew, that class's contribution to the aggregate is essentially one client's contribution. Encryption also dilutes nothing within a round in which the active learner has just added a handful of rare examples. We test this directly by attacking the decrypted aggregate in a "single-holder rare class" split. If reconstruction of rare-class acquisitions from the aggregate falls to chance, then encryption suffices for privacy, and our contribution narrows to the utility side. We would state that plainly.
Why not just add differential-privacy noise? Gradient Inversion with Generative Image Prior notes that noise strong enough to stop generative inversion costs main-task accuracy. Isotropic noise also falls hardest on the classes whose summed gradient is smallest, which are the rare ones. Our experiment matches DP to our method's attacker PSNR and compares minority recall at that fixed leakage level. If DP-SGD with per-class clipping matches us there, DP is the simpler answer.
Why not just make the calibration set class-balanced and stop? This is the most serious threat. It may recover most of the rare-class utility, and it costs almost nothing. The $\lambda = 0$ ablation settles it. If it matches the full method on accuracy and if, unexpectedly, it also reduces leakage, the dominance term is unnecessary and we will drop it. The full method has to earn its extra per-sample gradient cost through a measurable PSNR or SSIM reduction at equal accuracy.
Why not just prune harder, since sparsification is itself a mild defense? The constraint here is that GIAS reportedly reconstructs through 99% pruning. We include magnitude and random pruning at matching sparsity levels so that the claim "targeted pruning differs from generic pruning" is tested against exactly that result, not assumed.
Proposed Experiments
The datasets are a long-tailed CIFAR-10 in the style of the SIFAR-10 variant used by Mining Minority-class Examples With Uncertainty Estimates. We also use three naturally imbalanced sets from MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification (DermaMNIST, PathMNIST, Pneumonia-MNIST), with ResNet-18, which that benchmark finds competitive at 28×28 resolution. The third is CIC-IDS2017 for the intrusion setting. We sweep imbalance ratio and Dirichlet $\alpha$ as in LoGo, use labeling budgets from 1% to 20%, and use upload ratios from 0.1 down to 0.01.
Metrics are macro F1 and minority-class recall for utility, PSNR, SSIM, and label-recovery rate for privacy, and uploaded bytes plus client FLOPs for cost. Attacks are run per round against both individual and aggregated updates, using pixel-space matching and generative-prior inversion where a prior exists. We also run an active-attack variant in the style of OASIS: Offsetting Active Reconstruction Attacks in Federated Learning as a stress test outside our main threat model.
The decisive checks are three. First, the stage-one correlation must exist: without a positive link between acquisition score, reconstructability, and compression loss, the premise fails. Second, at equal bytes the full method must reduce attacker PSNR on rare-class acquisitions relative to $\lambda = 0$, by an amount larger than seed variance, without losing minority recall. Third, the method must beat discrepancy-aware calibration with random calibration on minority recall at the 0.01 ratio, the regime where that paper reports its largest gains over Top-$k$.
References
- Gradient Inversion with Generative Image Prior
- Re-thinking Federated Active Learning based on Inter-class Diversity
- Concealing Sensitive Samples against Gradient Leakage in Federated Learning
- Enhancing Communication Compression via Discrepancy-aware Calibration for Federated Learning
- Active Learning with Fast Model Updates and Class-Balanced Selection for Imbalanced Datasets
- EdgeDetect: Importance-Aware Gradient Compression with Homomorphic Aggregation for Federated Intrusion Detection
- Mining Minority-Class Examples with Uncertainty Estimates
- MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification
- OASIS: Offsetting Active Reconstruction Attacks in Federated Learning