Does Keeping the Classifier Private Actually Protect Data in Personalized Federated Learning?

Federated learning trains one model across many devices without collecting their data: each device computes an update on its own examples and sends only that update to a server. A popular way to cope with devices whose data look very different from one another is to split the network in two. The feature extractor is shared and averaged across devices. The final classifier, the "head", stays private on each device and is never sent. The personalization literature often treats the private head as a privacy bonus, on the reasoning that an attacker who never sees the head cannot read labels off it. The gradient-inversion literature, which studies how to reconstruct training images from transmitted updates, has almost never tested this setting. This note argues that the private head likely protects much less than assumed. More specifically, the standard recipe of fitting the private head first and then computing the shared update may make a few examples on each device easier to reconstruct than plain federated averaging would.

The mechanism is simple. When a device fits its private head well before computing the shared-feature update, most of its examples are already classified correctly and contribute almost nothing to that update. What gets transmitted is dominated by the handful of examples the head still gets wrong. These are typically the atypical, rare, and often most sensitive samples. Prior work shows that gradient inversion succeeds when per-sample contributions are not mixed together. A gradient effectively produced by one or two samples is the easy case. Bandwidth-saving compression may make this worse. Top-k sparsification keeps the largest coordinates, and those are exactly the coordinates belonging to the dominant samples. We propose to measure this effect directly and explain it. We then test a cheap fix, which caps how much any single example can contribute to the shared update, under matched communication budgets.

Core Claim

Consider the last shared layer, with weights $W_L$. For each example $i$ in a local batch, let $a_i$ be that layer's input and $\delta_i$ the error signal arriving at its output. This signal depends on the private head $h$ and on the residual $r_i = p_i - y_i$, the gap between predicted and true class probabilities. Every linear layer then satisfies

$$\nabla_{W_L} = \sum_i \delta_i a_i^\top, \qquad N_{\text{eff}} = \frac{(\sum_i \omega_i)^2}{\sum_i \omega_i^2}, \quad \omega_i = \|\delta_i\|\,\|a_i\|.$$

The first identity holds regardless of whether $h$ is visible. Hiding the head changes the scale and direction of each $\delta_i$, but each sample's term still has $a_i$ as its row direction. When one term dominates, the input to the last shared layer can be read off up to scale and then inverted back to pixels with a generative image prior. The quantity $N_{\text{eff}}$ is the effective number of examples contributing to the update. It equals the batch size when contributions are equal and approaches 1 when a single example dominates. Our claim has three parts. First, head-first decoupled training systematically lowers $N_{\text{eff}}$ relative to monolithic training at the same batch size, because fitting the head drives most $r_i$ toward zero. Second, reconstruction quality for the worst-case example tracks $N_{\text{eff}}$ more closely than it tracks batch size. Third, magnitude-based compression preserves the dominant examples' contribution rather than erasing it. If these hold, the decoupled protocols in Exploiting Shared Representations for Personalized Federated Learning and Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity need a privacy caveat. Batch size, the leakage-reducing knob that Evaluating Gradient Inversion Attacks and Defenses in Federated Learning identifies, would also be a misleading proxy in this regime.

Relation to Prior Work

Two papers supply the training protocols we attack. FedRep alternates between fitting a private head with the shared feature extractor frozen and then updating the extractor with the head frozen. It reports 87.7% personalized accuracy on CIFAR-10 with two classes per client, against 42.6% for federated averaging. FFGG proves that fine-tuning local parameters before taking the global gradient removes the slowdown caused by heterogeneous data. Both send only shared-parameter updates, and neither tests inversion. Our concern is aimed precisely at their head-first step, which is what makes them work.

Three papers motivate the leakage hypothesis. Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks shows that the last fully connected layer's gradient is a scaled copy of its input. That paper assumes the classifier gradient is visible. We extend the same algebra one layer down, to a shared layer whose downstream head is hidden. Concealing Sensitive Samples against Gradient Leakage in Federated Learning observes that reconstruction depends on low entanglement between sample gradients. $N_{\text{eff}}$ is our way of quantifying that entanglement and tying it to the head-fitting step. GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization searches a pretrained generator's intermediate feature spaces and reports 20.05 dB PSNR on ImageNet. It supplies the pixel-recovery stage of our attack.

Two papers constrain the design. MineGrad: Gradient Inversion Attacks on LoRA Fine-Tuning shows that sharing only a small fraction of the parameters does not protect privacy: up to 85.9% of CIFAR-100 patches are recovered at batch size 32. That attack, however, relies on a malicious server that poisons the frozen weights. We deliberately study the weaker, passive honest-but-curious server, because a leakage result there is harder to dismiss. Huang et al. show that attacks degrade with large batches, private batch-norm statistics, and unknown labels. We adopt all three relaxations so that the attacker is no stronger than in their realistic setting.

On compression, Personalized Federated Learning with Communication Compression compresses personalized updates without measuring privacy. Mixed Quantization Enabled Federated Learning to Tackle Gradient Inversion Attacks reports that private quantization scales drop DLG reconstruction from 55.3 dB to 8.1 dB, but only for monolithic federated averaging. Nobody has asked whether compression's protective effect survives when update energy is concentrated in a few samples.

Research Program

The work has three stages.

Analysis. We formalize when the input to the last shared layer is recoverable from $\nabla_{W_L}$ without knowing $h$. For a single dominant sample this follows from the rank-one structure. For a mixture we bound recovery error as a function of $N_{\text{eff}}$ and of the angles between the $a_i$. We also show why fitting the head to low training loss shrinks $N_{\text{eff}}$: in the overpersonalized regime FFGG analyzes, residuals vanish for all but the non-separable samples. We are not claiming a guarantee for deep networks. The analysis predicts which examples leak and how leakage should scale, and the experiments test those predictions.

Measurement. We build a passive attack tailored to hidden-head protocols. It first estimates the last-shared-layer inputs from the dominant directions of $\nabla_{W_L}$. It then inverts them to images with feature-domain generative search. As a check that does not rely on that first stage, it also runs a direct optimization in which the unknown head is a nuisance variable fitted jointly with the dummy images. The head is small (for example 512×10 weights), which makes joint fitting feasible. Across head-fitting steps, local batch sizes, data-skew levels, and compressors, we record $N_{\text{eff}}$ and per-sample reconstruction quality. Compressors include top-k, sign compression, QSGD-style quantization, and natural compression.

Fix. We test residual balancing. Before computing the shared update, the device rescales each example's error signal so that no example's $\omega_i$ exceeds a fixed quantile of the batch. This holds $N_{\text{eff}}$ near the batch size. It costs one extra per-sample norm computation, which is small next to the head-fitting loop the client already runs. It sends nothing extra and composes with any compressor. We are explicit that it is an empirical defense against the attacks we know, not a formal privacy guarantee, and we compare it against defenses that do carry guarantees.

Baselines

The naive reference point is undefended FedRep with full-precision shared updates, run exactly as in the original paper. This is the protocol whose assumed privacy we are testing. Its published 87.7% CIFAR-10 accuracy anchors utility, and its reconstruction quality under our attack is the number every defense must improve on. Next to it we run monolithic federated averaging at the same local batch size and compute. This isolates whether decoupling changes leakage at all. Huang et al. report LPIPS 0.45 with 93.37% accuracy for an undefended monolithic CIFAR-10 model at their settings, and we expect our monolithic arm to land in that range.

The strongest existing utility competitor is FFGG, the most heavily optimized head-first method. It reports 64.93% on Google Landmarks v2 against 51.43% for federated averaging. We run it with its published local-step schedule on CIFAR-10/100 and FEMNIST, plus a reduced GLDv2 adapter setting if compute allows.

The strongest existing privacy competitor at low bandwidth is mixed quantization with private per-layer scales. It reports the 55.3 to 8.1 dB DLG drop and a 4x payload reduction on monolithic CIFAR-10. We port it unchanged onto the shared-extractor update. We also include Huang et al.'s combined GradPrune 0.9 plus InstaHide defense, which reports 86.10% accuracy and LPIPS 0.73 on CIFAR-10 at batch size 32. Finally, we include DP-SGD with per-sample clipping and Gaussian noise, tuned to the same accuracy target.

Every method receives the same client partitions, local compute, number of rounds, and hyperparameter tuning budget. That budget is a fixed grid of equal size per method, selected on validation accuracy before any attack is run. Communication is matched in bits per round, and compressors are applied identically.

Two ablations isolate the key ingredient. The first varies the number of head-fitting steps from zero up to FedRep's default, with nothing else changed. This tests whether head fitting itself drives leakage. The second applies per-sample clipping without noise, which is the closest existing operation to residual balancing.

To count as beating the baselines, residual balancing must do one of two things. Either it matches undefended FedRep's personalized accuracy within one point while reducing the fraction of examples reconstructed below an LPIPS threshold more than mixed quantization does at equal bits. Or it matches DP-SGD's worst-case leakage at clearly higher accuracy.

Why not just…?

Why not just add differential-privacy noise? It gives the only formal guarantee here, so it is a real competitor. The file's evidence is that it is expensive in accuracy. In the mixed-quantization paper, DP noise held MNIST at roughly 78–85% accuracy against 96.67% for quantization at comparable security. The hard-label paper finds analytic inversion survives until noise variance reaches about $10^{-1}$, where utility is likely badly damaged. Personalized accuracy on skewed CIFAR may be more forgiving, because the head is trained on clean local data. We therefore report the full accuracy–leakage curve for DP-SGD rather than a single point. If DP-SGD dominates residual balancing on that curve, we will recommend DP-SGD and present our contribution as the leakage diagnosis alone.

Why not just compress harder, since pruning and quantization already hurt inversion? Huang et al. found that pruning alone must exceed 99% of coordinates to stop attacks, at about 10% accuracy loss, and GIFD and PEFTLeak both survived sparsification. Our specific prediction is worse for this alternative. When one sample dominates, top-k keeps its coordinates, so compression should protect decoupled training less than it protects monolithic training. The compressor sweep tests this directly. If compression alone suppresses leakage in decoupled training as well as residual balancing does, the fix is unnecessary and we will say so.

Why not just take fewer head-fitting steps? This is the simplest knob, and it may be enough. Fewer head steps keep residuals large and $N_{\text{eff}}$ high, but FedRep's and FFGG's accuracy gains depend on fitting the head before the shared update. The head-step ablation traces the accuracy–leakage frontier of this knob. Residual balancing earns its place only if it reaches a better point on that frontier. Otherwise we adopt fewer head steps as the recommendation.

Why not just use secure aggregation? Secure aggregation, which lets the server see only the sum of a cohort's updates, does defeat a passive server looking at individual devices. It is complementary to our work rather than a substitute. ERIS and the mixed-quantization paper both note its computational and key-management costs at scale. The aggregated sum over a small cohort can itself have low effective sample count when each device contributes one dominant example. We test the sum over cohorts of 2, 5, and 10 devices to show how much of the effect survives aggregation.

Proposed Experiments and Decisive Checks

We use CIFAR-10 and CIFAR-100 with FedRep's label-skew partitions (2 and 5 classes per client for CIFAR-10, scaled up for CIFAR-100), FEMNIST from LEAF: A Benchmark for Federated Settings with its natural per-writer split, and a reduced GLDv2 adapter configuration for FFGG. Local batch sizes are 8, 32, and 64. Batch-norm statistics stay private and labels are unknown to the attacker. We report PSNR, SSIM, and LPIPS averaged over examples and for the worst example per batch, plus the fraction of examples below LPIPS 0.3. We also report personalized test accuracy, bits uploaded per round, and the measured $N_{\text{eff}}$ of every transmitted update.

The first decisive check is whether worst-example reconstruction in FedRep and FFGG is better than in monolithic averaging at the same batch size, and whether $N_{\text{eff}}$ predicts reconstruction quality better than batch size does. We assess this by rank correlation across all configurations. A null result here, meaning the private head does protect, would itself be worth publishing, since nobody has measured it.

The second check is whether, at matched bits, top-k and sign compression preserve worst-example leakage in decoupled runs while suppressing it in monolithic runs.

The third check is whether residual balancing pushes worst-example LPIPS above that of mixed quantization and of the noise-free clipping ablation, within one point of undefended personalized accuracy.

We also verify that the attack does not depend on the first-stage estimation trick. To do this, we report the joint head-and-image optimization attack separately. Any leakage it finds cannot be attributed to our rank-one estimation step.