Which Coordinates a Client Sends Can Reveal What Its Data Contains

Federated learning trains one model across many devices without collecting their data. Each device computes an update on its own examples and sends only that update to a server. Because full updates are large, practical systems compress them. A common approach is sparsification: the client sends only a few coordinates, chosen by magnitude or by how much each one changes the model's outputs on local data. A separate line of work hides the values of updates from the server, using secure aggregation (the server sees only a sum), locally scaled quantization, or encryption of selected parameters. This note argues that these two lines of work combine into a leak neither has examined. When clients hold different data distributions, the set of coordinates each client chooses to send depends on that client's data. The index pattern is visible even when every value is hidden, so it can reveal the client's label mix or domain.

The opportunity has two parts. First, measure a side channel that current value-hiding defenses leave open. Second, test whether compression can keep the accuracy gains of data-aware selection while making the selection itself independent of any one client's data. If the side channel is real, several recent systems that treat the choice of transmitted parameters as harmless metadata would need to rethink that assumption.

Core Claim

Our claim is that data-dependent sparsification masks leak the client's data distribution, and that the leak grows with statistical heterogeneity (how differently data is distributed across clients). Write the transmitted index set for client $i$ as $M_i = \mathrm{TopK}(s(\Delta_i; D_i))$, where $\Delta_i$ is the local update, $D_i$ the local data, and $s$ a scoring rule (magnitude, output discrepancy, or saliency). The hypothesis is that the mutual information $I(M_i; p_i)$, between the mask and the client's label proportions $p_i$, is large under non-IID splits even when all values in $\Delta_i$ are concealed. We also expect it to stay large late in training, when update values have shrunk.

The final layer gives the clearest intuition. Decaf is a passive attack that identifies classes absent from a client's data with 100% accuracy, because those classes' output neurons receive non-positive updates. It then estimates the remaining class proportions within 5% $L_\infty$ error. Decaf needs the update values. We conjecture that a top-$k$ rule preferentially selects the rows of classes the client actually has, so the index pattern alone reproduces much of Decaf's signal. Decaf also reports that its accuracy degrades in late rounds as gradients converge. A ranking can remain stable after magnitudes collapse, so the mask may outlive the value channel.

Discrepancy-aware selection makes this sharper. Enhancing Communication Compression via Discrepancy-aware Calibration ranks coordinates by the output shift they cause on 32–64 local calibration samples, and beats magnitude top-$k$ on ViT-tiny/CIFAR-10 at a 0.01 compression ratio (29.62% vs. 21.03%). That paper explicitly leaves privacy unexamined. Its selection rule is, by design, a function of which local examples are most sensitive, which is exactly the property that would make a mask informative.

SHE-LoRA encrypts only the columns of low-rank adapters (LoRA, small trainable matrices added to a frozen model) that Wanda saliency marks as sensitive. It negotiates which columns to encrypt using order-preserving encryption, and it defeats the DAGER inversion attack while encrypting as little as 0.03% of parameters. Its encrypted values are safe, but which columns a client flags is itself a saliency-derived, data-dependent choice. The authors note this negotiation may fail or leak when tasks differ sharply.

The defensive half of the claim is that a mask computed only from shared state can recover most of the accuracy advantage of data-aware selection. By shared state we mean the current global model, the previous aggregated update, and a public calibration set. Error feedback would keep the discarded local residual on the device.

Research Program

The first component is an attack on masks alone, under a threat model where the server sees index sets but not values. This covers secure aggregation of sparse vectors, where indices must be coordinated or revealed. It also covers value quantization with locally held scales, as in Mixed Quantization Enabled Federated Learning, and selective encryption. We build two attackers.

The first is analytic. For the final classifier layer, it predicts absent classes from unselected rows and estimates class proportions from per-row selection frequency across local steps and rounds, mirroring Decaf's logic without values. The second is learned. The server simulates shadow clients on proxy data with known label mixes or domains and trains a small model that maps masks from all layers to the label histogram or domain identity. The use of proxy data follows the server capability assumed by MineGrad. We keep the attacker passive throughout, because active manipulation of the model already defeats most defenses, as MineGrad and Breaking Secure Aggregation show. A passive leak is the more damning finding.

The second component is a leakage-versus-heterogeneity curve. We sweep Dirichlet label skew on CIFAR-10 and CIFAR-100, and feature shift on the Digits benchmark used by FedBN. For each scoring rule and compression ratio, we measure how much mask leakage exceeds a random-mask floor. The prediction is that leakage rises with skew, is largest for discrepancy-aware and saliency rules, and exceeds what Decaf recovers from values in late rounds.

The third component is shared-state masking. Each round, the server broadcasts a selection computed from the global model's sensitivity on a public calibration set and the previous aggregate. Every client in a round, or in a server-assigned cohort, uses the same mask. Each client adds its unsent residual to the next round's update (error feedback), as studied in Rethinking gradient sparsification as total error minimization. Because the mask is fixed before local training, it carries no information about $D_i$ beyond what the global aggregate already carries. The research question is how much accuracy this costs relative to per-client discrepancy-aware selection under strong heterogeneity. Per-client selection is exactly what adapts to local data, so there is a genuine tension here.

We will study LoRA-based fine-tuning of a CLIP vision transformer on OfficeHome as the foundation-model setting, where sparsification is applied to adapter factors. This connects the work to SHE-LoRA and to FedBip, which clusters clients by task-vector similarity and does not examine privacy.

Baselines

The naive reference point is uncompressed FedAvg with values visible to the server, attacked by Decaf. This provides the value-channel upper bound on distribution leakage: Decaf reports under 5% $L_\infty$ error and 100% absent-class identification on CIFAR-10. We re-run Decaf on our own splits rather than importing those numbers. The random-mask floor is shared-seed random-$k$ sparsification. Its mask is independent of data by construction, so any attacker success on it measures leakage from priors alone.

The strongest existing compression method is discrepancy-aware calibration. The reported result to beat on utility is 29.62% accuracy for ViT-tiny on CIFAR-10 at a 0.01 compression ratio, against 21.03% for magnitude top-$k$. We run it with the authors' 32–64-sample calibration budget, the same client count, local steps, and round budget as our method, and an equal hyperparameter sweep for every compressor.

The strongest existing value-hiding defense is mixed quantization. It reports DLG reconstruction PSNR falling from 55.3 to 8.1 and 96.67% accuracy on MNIST, with a 4x payload reduction. We pair it with top-$k$ selection to show that hiding values does not hide masks. For LoRA experiments, SHE-LoRA's saliency-based column selection is the saliency-driven comparator. Its column-selection leakage has, to our knowledge, not been measured, so we report whatever we observe.

The key ablation isolates where the mask comes from. We hold the scoring rule and the error feedback fixed and switch the input to selection between local data and shared state. We also try per-cohort masks as an intermediate point.

Shared-state masking beats the field if it meets two conditions at matched bytes. On privacy, mask-only attack error must be statistically indistinguishable from the random-mask floor. On utility, it must recover at least half of the accuracy gap between random-$k$ and per-client discrepancy-aware selection under Dirichlet $\alpha \le 0.1$.

Why not just…?

Why not just use random sparsification with a shared seed? It removes the side channel entirely and is the real threat to the defensive half of this proposal. The evidence that data-aware selection matters comes from the discrepancy-aware paper, where magnitude top-$k$ already loses 8.6 points to calibrated selection at 0.01. The file reports no random-$k$ number, so its gap is unmeasured here. If random-$k$ with error feedback matches shared-state masking in our runs, we will say so and recommend it. The attack contribution stands regardless, and the practical conclusion becomes simply "do not let local data choose the indices."

Why not just add differential privacy noise? Noise on values does nothing to a mask chosen before noise is added. Noise large enough to scramble the ranking is costly. Mixed Quantization reports DP baselines dropping MNIST accuracy to about 78–85%, and ExPLoit reports a 38% accuracy loss from gradient noise on CIFAR-10. We include randomized selection of indices as a DP-style baseline and measure its accuracy cost at matched attack error.

Why not just rely on secure aggregation? Sparse secure aggregation still needs agreement on which indices are summed, which is the channel we attack. Under full dense secure aggregation, the communication savings disappear. Separately, Breaking Secure Aggregation shows an active server recovers labels with 100% accuracy through a manipulated normalization layer. Secure aggregation is therefore not a sufficient guarantee on its own. Our experiments quantify the passive, mask-only leak that remains even against an honest server.

Why not just keep the classifier head local, as in personalized methods? Keeping the head private, as FedRep does, removes the layer Decaf exploits. Label skew may still imprint on which body coordinates are selected, so we run the mask attack on FedRep-style body-only masks as an explicit condition. If the leak vanishes there, that is a useful and publishable negative result.

Proposed Experiments

The first decisive check is mask-only distribution inference on CIFAR-10 with 100 clients under Dirichlet $\alpha \in \{0.05, 0.1, 0.5, 100\}$. We compare magnitude top-$k$, discrepancy-aware selection, and random-$k$ at compression ratios 0.1 and 0.01, with values hidden by locally scaled quantization. We report absent-class identification accuracy and $L_\infty$ proportion error, and compare both against Decaf with values visible, round by round.

The hypothesis is falsified if discrepancy-aware masks stay within noise of the random floor at $\alpha = 0.1$. It is strongly confirmed if mask-only error in late rounds falls below Decaf's value-based error at the same rounds.

The second check applies domain identification on Digits and OfficeHome. The attacker predicts which domain a client's data comes from using LoRA-factor masks and SHE-LoRA-style saliency column choices.

The third check measures the utility of shared-state masking against per-client discrepancy-aware selection and random-$k$ with error feedback, at equal bytes and rounds. We use CIFAR-10 (ViT-tiny) and OfficeHome (CLIP-ViT with LoRA). Each configuration runs with three seeds, and we report both personalized and global accuracy.