Metadata-Conditioned Gradient Inversion under Differential Privacy

When a federated learning system defends its model updates with differential privacy — adding calibrated Gaussian noise so that no single client's data can be reconstructed from what the server sees — it usually leaves the plumbing unprotected. To keep decentralized training converging on uneven data, clients also send coordination signals in the clear: label distributions, batch normalization statistics, participation counts, loss trajectories. This note proposes to test a specific and, so far, unexamined failure mode: an honest-but-curious server can use these plaintext metadata signals as constraints inside a gradient inversion attack, sharply reducing the search space the noise has to obscure. If true, the noise scale that practitioners consider safe, calibrated per update under standard accounting, is not safe in any real system that also shares metadata — and clients are paying a utility cost for privacy they are not actually getting.

The core mechanism is easy to state. Gradient inversion reconstructs private training inputs by finding dummy data whose gradients match the observed ones; generative attacks like Gradient Inversion with Generative Image Prior search a generator's latent space instead of pixel space, which is what makes them robust to large batches and sparsified updates. Metadata narrows that search further for free. The update to the final classification layer's bias is, as Heterogeneity-Guided Client Sampling showed, an exact readout of the client's class proportions; batch normalization means and variances pin down the first two moments of the client's feature distribution, which is precisely why synchronizing them rescues non-IID training in FedTAN; participation telemetry lets an attacker un-mix aggregated updates entirely, as Gradient Disaggregation demonstrated. Each of these papers treats its signal as either a vulnerability, a coordination tool, or an attack enabler in isolation. None asks what happens when the server holds both a DP-noised update and the plaintext metadata that describes it.

Core Claim

The claim is two-sided. On the attack side: conditioning generative gradient inversion on plaintext coordination metadata (class priors, batch statistics, participation structure) restores usable reconstruction fidelity at DP noise scales substantially larger than those at which the unconditioned attack fails. On the defense side: the honest fix is not more noise on the model update but a co-allocated privacy budget that treats metadata as a DP release in its own right, so the total guarantee accounts for every channel the server actually observes. Both sides are measurable with existing attacks, existing benchmarks, and existing accounting tools; what is missing is the connecting experiment and the erosion quantification.

The distinction from the closest prior work matters here. Gradient Disaggregation uses participation metadata to solve a linear algebra problem — recovering the participant matrix and un-mixing an aggregate — and explicitly does not analyze what happens when DP noise breaks that linear structure. LIA-SA recovers labels perfectly but only as an active adversary who poisons batch normalization parameters and distributes crafted models; it never analyzes formal $(\epsilon,\delta)$ bounds or how DP clipping bias distorts its pseudo-inverse solver. MineGrad shows low-rank adapters don't block inversion, but again through active backbone poisoning and against heuristic noise, not calibrated DP. This proposal is passive (the server changes nothing and can be audited), operates against formally accounted DP noise, and locates the leakage in the interaction between two channels rather than in either one alone.

Research Program

The first component builds the attack, which we can call metadata-conditioned inversion. Starting from the generative-inversion recipe of GIAS/GIML — optimize a latent code $z$ in a pretrained generator $g_\phi$, then fine-tune the generator itself — we add the observed metadata as hard or soft constraints on the candidate samples. Writing $\hat{G}$ for the DP-noised update the server observes and $c_m(\cdot)$ for the server-side function that extracts a metadata value $m$ (class prior from the final-layer bias, feature moments from BN statistics), the attack solves

$$ \min_{z,\,\phi}\;\; \big\|\nabla_\theta L(g_\phi(z);\theta) - \hat{G}\big\|_2^2 \;+\; \lambda \sum_{m \in \mathcal{M}} \big\| c_m(g_\phi(z)) - m \big\|_2^2, $$

where $\mathcal{M}$ is the set of plaintext metadata signals, $\lambda$ balances the terms, and GIML's meta-learning variant supplies the generator prior when no pretrained model exists. In a secure-aggregation setting, the Gradient Disaggregation pipeline (whose code is public at gdisag/gradient_disaggregation) first isolates per-client updates from participation metadata, and the conditioning then applies to the isolated gradient. Two properties of DP noise make the conditioning plausibly decisive: Gaussian noise is isotropic and dimension-proportional, so it is weakest exactly where metadata has already pinned down structure, and clipping bias — the distortion documented in Adaptive Clipping for Differential Private Federated Learning in Interpolation Regimes — is itself a systematic, partially predictable transformation the attacker can model.

The second component is the measurement we actually care about: an erosion factor. For each metadata configuration, we sweep the DP noise scale and record reconstruction fidelity (peak signal-to-noise ratio, patch recovery rate, label recovery accuracy) with and without conditioning. The erosion factor $\rho$ is the ratio of the largest noise scale at which the unconditioned attack still succeeds to the largest scale at which the conditioned attack succeeds; $\rho = 1$ means metadata adds nothing and the claim is falsified. We also translate $\rho$ into accounting language: the effective privacy loss of the joint release (noisy update plus plaintext metadata) versus the nominal $\epsilon$ of the update alone. This is the quantity practitioners currently have no way to estimate.

The third component is the defense response: co-allocated DP. Metadata must be privatized or suppressed, and the budget must compose. Under Rényi differential privacy accounting — the same machinery used by AdaptDP-FedAvg for its noisy clipping radii — the total loss of the joint release composes as

$$ \epsilon_{\text{total}}(\alpha) \;=\; \epsilon_{\text{model}}(\alpha) \oplus \epsilon_{\text{meta}}(\alpha) \quad \text{for each RDP order } \alpha, $$

and the research question is where to place the split. Privatising class-prior metadata with the Laplace or Gaussian mechanism is cheap because the released statistic is low-dimensional (the $O(C)$ bias readout of HiCS-FL); privatising BN statistics is expensive because, per FedTAN, those statistics carry most of the coordination value on non-IID data. The program here is empirical rather than a new theory of optimal splits: measure the utility curve (federated test accuracy and rounds-to-target) and the attack curve (reconstruction fidelity) as the split varies, and characterize the frontier. Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD warns that worst-case DP demands can destroy utility; the frontier measurement is how we find out whether any split is simultaneously livable and safe.

Baselines

The naive reference point is plain DP-FedAvg with fixed clipping and no metadata released at all: it defines the utility ceiling and the privacy floor, and it is the configuration every DP deployment implicitly claims to be. It will be run under the same non-IID partitions, the same round budget, and the same Rényi accounting as every other configuration.

The strongest attack competitor is unconditioned GIAS/GIML from Gradient Inversion with Generative Image Prior, which is the best inversion result in the file: it reports a reconstruction PSNR of 15.58 dB versus 13.78 dB for pixel-space attacks, maintains that fidelity at batch size 32 where standard attacks need batch size 4, and still reconstructs images when 99% of gradient values are pruned. Our attack is its direct extension, so the comparison isolates exactly one ingredient — the metadata terms in the objective — and will be run with the identical generator, identical latent dimension, identical data, and an equal optimizer-step budget, with conditioning given no extra compute. The result that counts as beating it is a measurable rightward shift of the failure threshold: the conditioned attack achieving comparable fidelity at a DP noise scale at least $\rho$-fold larger, with $\rho$ reported per metadata channel. As a secondary attack reference, Deep Leakage from Gradients (reconstruction MSE of roughly 0.0069 on CIFAR-100 in its original setting) anchors the optimization-based, small-batch regime.

On the defense side, the strongest existing competitor is AdaptDP-FedAvg, which adapts the clipping radius from privately transmitted RMS gradient norms and reports lower training loss than fixed-clipping DP-FedAvg at identical $(\epsilon, \delta)$ budgets, with an asymptotic error bound of $o(\epsilon_{util}^{DP-SGD})$; no absolute accuracy figure is reported on Lacuna, so we will reproduce it ourselves under our partitions and treat our reproduction as the reference number. Our defense will be evaluated against it under the same total budget, and the comparison is fair only if the co-allocated system is given the same tuning budget for its metadata mechanisms that AdaptDP gets for its clipping schedule. Finally, the key ablation is the attack run with each metadata channel disabled in turn — class priors only, BN statistics only, participation structure only — because the claim is not that metadata helps in general but that specific, commonly shared signals each carry a quantifiable share of the erosion.

Why not just…?

Why not just use Secure Aggregation with large cohorts? Because disaggregation attacks already defeat it, and metadata is what powers them. Gradient Disaggregation recovered the participant matrix with 100% accuracy for over 1,000 users from participation-count telemetry alone, lifting inversion PSNR from 6.1 dB on the aggregate to 36.5 dB and property-inference AUC from about 0.65 to above 0.9; LIA-SA recovered labels with 100% accuracy through secure aggregation at batch sizes up to 5,120. Cryptography hides individual gradients; it does not hide the metadata that describes them.

Why not just stop sharing metadata? Because the coordination value is real and large, and this proposal's defense experiments would say so if suppression were the answer. FedTAN shows that uncoordinated batch normalization drives federated ResNet accuracy on extremely non-IID CIFAR-10 down to 45.9%, while synchronizing BN statistics restores it to 87.6% against a centralized ceiling of 91.5%; DWFed cuts communication rounds from 800 to 520 using plaintext label distributions. Suppression is a baseline we will run, and if the utility loss turns out to be tolerable at the noise levels metadata-conditioned attacks demand, that is a legitimate finding — but the 42-point accuracy gap above suggests it will not be.

Why not just add more noise to the model update? This is the most serious objection, because standard composition is the textbook answer: privatize the metadata too and compose the losses, which is close to what our defense component does. Two things keep it from being the whole answer. First, no existing work measures whether metadata conditioning actually erodes empirical DP or by how much — GIAS notes noise defeats generative inversion at sufficient scale, but never with metadata in the attacker's hands, and AdaptDP-FedAvg evaluates utility only, never attack resistance — so the required noise level, and hence the utility price, is unknown. Second, Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD establishes that worst-case budgets are often utility-destroying, so the interesting question is not whether composition is formally correct but whether any split of a fixed total budget is simultaneously livable and safe. Our experiments are designed to answer exactly that, and if naive "noise everything equally" matches a smarter split on the frontier, we will report it.

Why not just use gradient pruning, FP16, or large batches instead of DP? The file's evidence rules these out as primary defenses. GIAS reconstructs images under 99% gradient sparsification; DLG found half-precision does not stop leakage and that noise only defends at variances above $10^{-2}$, which gutted accuracy; LIA-SA works at batch sizes of 5,120. Heuristics give no accounting, so they cannot even express the question this proposal asks.

Proposed Experiments and Decisive Checks

The testbed uses CIFAR-10 and CIFAR-100 with Dirichlet-partitioned non-IID clients, CelebA and FEMNIST from the LEAF benchmark for realistic user-keyed heterogeneity, and ResNet/VGG architectures with batch normalization to match the settings of LIA-SA and FedTAN. Training runs FedAvg and FedTAN-style BN synchronization under DP-FedAvg with Rényi accounting across a grid of noise scales; the attacker is honest-but-curious, receiving the noisy aggregate, the plaintext metadata channels, and, in secure-aggregation runs, the participation telemetry consumed by the public Gradient Disaggregation code. Attack metrics are reconstruction PSNR and MSE, patch recovery rate (to compare against MineGrad's 85.9% on CIFAR-100 at batch size 32, noting ours is passive where theirs poisons the backbone), label recovery accuracy, and property-inference AUC; utility metrics are final test accuracy and rounds to target, following the DWFed protocol.

Three checks decide the proposal. First, the erosion sweep: if $\rho \approx 1$ across all metadata channels and noise scales — conditioning never moves the failure threshold — the core claim is false and the honest conclusion is that per-update DP accounting is empirically robust to metadata side-channels, which would itself be a useful negative result against the intuition built by Gradient Disaggregation and HiCS-FL. Second, the frontier: whether any co-allocated split achieves both attack failure at its own noise levels and utility within a few points of the 87.6% BN-synchronized reference, or whether the frontier is empty — the outcome Fundamental Limitations would predict. Third, the transfer check: whether erosion factors measured against GIAS also hold against DLG-style optimization attacks and against the noisy-clipping pipeline of AdaptDP-FedAvg, since a claim about DP noise in general must survive the defender choosing the best available clipping schedule. No reported result exists on Lacuna for any conditioned-inversion-under-DP experiment, which is the gap this proposal fills.