In federated learning, many devices train a shared model by sending only parameter updates — gradients — instead of raw data, on the theory that a gradient is too abstract to reveal the data behind it. That theory fails: an adversary who sees a gradient and knows the current model weights can run an optimization that searches for the input which would have produced exactly that gradient, effectively reconstructing the private training image. Defenses exist — pruning the leakiest parts of the update, quantizing it to coarse integers, padding it with decoy gradients — but every published defense is a fixed transformation, and every published evaluation pits it against a fixed attack run once. Meanwhile the attacks themselves are becoming learning systems: the GIML attack of Gradient Inversion with Generative Image Prior meta-learns the client's data distribution from scratch across successive training rounds, so the attacker the defense faces at round fifty is much stronger than the one it faced at round one.
Security research outside of machine learning learned long ago that this situation is a game, not a fixed benchmark. As shown for wildlife and infrastructure protection in Learning and Planning Under Uncertainty for Green Security, the act of deploying a defense changes the adversary's behavior: opponents surveil defensive patterns and shift to whatever is left unguarded. A predictable defense is a computable counter-strategy. This proposal imports that lesson into gradient privacy. The client becomes a leader who commits to a randomized, time-varying defense schedule — rotating among quantization modes, pruning masks, decoy subspaces, and noise directions across communication rounds — and the inversion attacker becomes a follower who observes the resulting stream of shared gradients and best-responds, possibly by meta-learning. The claim is not merely that randomization helps; it is that the schedule can be planned tractably, because a closed-form leakage metric exists that scores a candidate defense configuration hundreds of times faster than running the attack itself.
Core Claim
Static gradient defenses fail against learning attackers primarily because they are stationary, not because their per-round transformations are weak. Formally, the client picks a policy $\pi$ that maps the history $h_t$ of past configurations and shared updates to a distribution over defense configurations $c_t$ (a choice of quantization bit-widths and modes, a representation-pruning mask, a decoy-gradient subspace, or a noise placement), and solves
$$\min_{\pi}\; \sum_{t=1}^{T} \mathbb{E}_{c_t \sim \pi(\cdot \mid h_t)}\big[\ell_t(c_t;\hat{A})\big] \quad \text{subject to} \quad \mathrm{Acc}(W_T) \geq \mathrm{Acc}_{\text{plain}} - \delta,$$
where $\ell_t(c_t;\hat{A})$ is the round-$t$ leakage cost of configuration $c_t$ against the anticipated attacker $\hat{A}$, estimated with the influence-function metric described below, and $\delta$ caps the accuracy loss relative to undefended federated learning (we use $\delta = 1.5\%$, the utility budget OASIS reports as its maximum drop on CIFAR-100). The core empirical claim: at matched utility, a planned moving-target schedule holds reconstruction quality substantially lower — measured as attack success integrated over all rounds — than any fixed configuration and than uniformly random switching, when the attacker is allowed to adapt across rounds.
Two facts from the literature make planning plausibly better than blind randomization. First, attack strength is time-varying: GIML accumulates distributional knowledge over rounds, while the systematization in SoK: Gradient Leakage in Federated Learning notes that gradients become less informative as training converges, so vulnerability is front-loaded and the utility budget should be spent early. Second, A Receding-Horizon MDP Approach for Performance Evaluation of Moving Target Defense in Networks shows that defense-shuffling frequency has a non-monotonic effect on security — reshuffling too often can actively help the attacker escape trapped states — so an unexamined random schedule at a fixed frequency is not obviously safe.
Research Program
The first workstream formalizes the repeated leader-follower game. The client (leader) commits to a mixed strategy over defense configurations each round; the server (adversary) observes the induced gradient stream and best-responds with an inversion procedure drawn from the published arsenal: generative-prior inversion (GIAS), round-by-round meta-learning (GIML) from Gradient Inversion with Generative Image Prior, fast transcript-based reconstruction (GIT) from Gradient Inversion Transcript, and participant-level model inversion with a diversity regularizer from FedInverse. The defender's solution concept follows the Stackelberg tradition of Robust Protection of Fisheries with COmPASS: commit to the randomized strategy that is optimal against the attacker's best response, not against a frozen attack snapshot. This is the exact inversion of how current defenses are built — Soteria prunes a fixed set of leaky representation elements, DCS² projects onto a fixed decoy subspace, and Mixed Quantization assigns fixed bit-widths per layer — all optimized against a stationary attacker.
The second workstream makes the game solvable. Evaluating a candidate configuration by running iterative inversion inside the planning loop is hopeless: GIT reports iterative methods taking thousands of seconds per batch. Instead we use the inversion influence function ($I^2F$) of Understanding Deep Gradient Leakage via Inversion Influence Functions, a closed-form Jacobian-based estimate of reconstruction risk that the authors report computing over 500× faster than running a standard attack, and which already correlates linearly with empirical reconstruction quality on language models. Crucially, $I^2F$ assumes a perfect attacker that reaches the global optimum of the inversion loss, while our attacker is bounded and prior-driven — so a central deliverable is a calibration study: does $I^2F$ rank defense configurations correctly when the true attacker is GIML or GIT rather than the idealized one? Where it misranks, we fit a correction term from a small set of ground-truth attacks. $I^2F$ also supplies defense actions directly: it shows noise aligned with small singular values of the Jacobian protects best, so directional noise placement becomes one of the configurations the policy rotates.
The third workstream solves for the policy. We start with a receding-horizon controller over a discretized configuration space (small enough to enumerate at each round) and a bandit-style variant for larger spaces, with an explicit switching-cost term motivated by the non-monotonic frequency findings of Receding-Horizon MTD. The fourth workstream is a robustness question imported from Imitative Attacker Deception in Stackelberg Security Games: if the client's leakage estimator is observable or manipulable, a strategic attacker can feed it false signals and trap the client in a predictable defensive posture (the "Maximin trap" Nguyen and Xu prove in abstract games). In our setting the manipulation channel is concrete — an active malicious server of the RTF/CAH type that OASIS studies tampers with the weights the client computes gradients on, thereby shifting the client's $I^2F$ reading. We will test whether deterministic defender policies are exploitable this way and whether randomized commitment, the classical Stackelberg answer, closes the trap.
Baselines
The naive reference point is undefended federated learning, where reconstruction is essentially solved: the DCS² paper reports peak signal-to-noise ratio (PSNR, an image-similarity measure where higher means a more faithful reconstruction) of 59.20 on MNIST with no defense. Any defense must first crush that number, but the informative comparison is what survives an adaptive attacker.
The strongest existing methods are the best static defenses at matched utility. Mixed Quantization reports driving the DLG attack's PSNR on MNIST from 55.3 down to 8.1 while keeping 96.67% model accuracy, and DCS² reports PSNR of roughly 7.84 on MNIST and structural similarity (SSIM) of 0.15 on CIFAR-10, at 65.68% test accuracy on TinyImageNet versus Soteria's 52.06%. We will run both, plus Soteria and OASIS, under identical conditions: same datasets, same number of clients and rounds, same attacker compute budget, and hyperparameter grids sized so every method gets an equal tuning effort and equal communication overhead. The decisive caveat is that these static numbers were earned against one-shot attacks; FedInverse shows static defenses tuned on one dataset fail completely on another, with diversity-regularized attackers reaching 100% Top-5 reconstruction accuracy within a few rounds. So the bar for beating the strongest baseline is: under the full adaptive attacker suite, hold cumulative attack success (area under reconstruction quality and identification accuracy over all rounds) meaningfully below the static defenses' trajectory at equal or better final test accuracy — not merely match their single-round PSNR.
The key ablation isolates the proposal's two ingredients separately. Uniformly random switching over the same configuration set, with no planning, tests whether game-theoretic scheduling adds anything beyond unpredictability itself. A second ablation replaces the $I^2F$ reward with the true (slow) attack-in-the-loop reward on small scale, testing whether the cheap surrogate picks the same schedules; the $I^2F$ code is public per the I²F paper, and we will build on the released FedInverse Implementation Codebase and Gradient Disaggregation Codebase for the attacker suite.
Why not just…?
Why not just add differential-privacy noise? Because the cost is documented and the protection is directional-blind. The Mixed Quantization paper reports that DP had to degrade MNIST accuracy to roughly 78–85% to reach protection that quantization achieved at 96.67%, and GIAS reconstructs images even when 99% of gradient values are removed. That said, DP is a real threat to this proposal's value, and we treat it accordingly: isotropic noise at several scales is one of the comparison arms, and directional noise aligned with small Jacobian singular values — which the $I^2F$ paper shows is where perturbation protects most — is one of the configurations our policy rotates. If fixed directional noise alone matches our full schedule, the scheduling machinery is unjustified and we will say so.
Why not just quantize to 2 bits and take multiple local SGD steps? The systematization in SoK shows these cheap measures already break most academic attacks at under 5% accuracy loss. This is the most serious objection, and the honest answer is that it may hold against today's published attackers while failing against the attackers this proposal builds: GIML accumulates distributional knowledge across rounds regardless of per-round obfuscation, and FedInverse-style participant-level inversion sidesteps intercepted-gradient fragility entirely. The decisive experiment is explicit: a 2-bit-plus-multi-step baseline run against our strongest adaptive attacker over 100 rounds. If that baseline holds, the proposal fails on its own terms and the negative result gets reported.
Why not just randomize the defense every round? Uniform randomization is a mixed strategy and may capture most of the benefit; we do not argue around this, we test it as the primary ablation. The concrete reasons to expect planning to matter are mechanistic, not rhetorical: attack strength grows with rounds (GIML) while gradient informativeness shrinks near convergence, so a stationary random schedule wastes utility budget when the attacker is weak and under-defends when it is strong; and the non-monotonic switching-frequency effect found by Qian et al. means the frequency and timing of switches, not just their unpredictability, carry security weight. If uniform random matches the planned policy across datasets, the contribution reduces to "randomization suffices against adaptive inversion," which we would report as such.
Proposed Experiments and Decisive Checks
Experiments run federated learning with 100 non-IID clients (two classes per device, matching Soteria's setup) on MNIST, CIFAR-10, CIFAR-100, TinyImageNet, CelebA, and ImageNet at 64×64, with LeNet-5 and ResNet-family targets over 100 rounds. The attacker suite is DLG as the weak reference, GIAS/GIML, GIT (which reports CIFAR-10 reconstruction MSE of 0.010 versus DLG's 0.073, at roughly 0.002 seconds per batch — fast enough to serve as a high-throughput adaptive attacker), and FedInverse's GMI/KED-MI/VMI variants with the HSIC diversity regularizer, whose KED-MI variant already reports 80% identification accuracy on undefended MNIST. For the deception workstream we add active RTF/CAH servers. Metrics are PSNR, SSIM, and MSE of reconstructions; Top-1/Top-5 identification accuracy per FedInverse; final model test accuracy; and cumulative leakage over rounds.
Five checks decide the proposal. First, the headline comparison: planned schedule versus best static defense versus uniform random versus no defense, all under the adaptive attacker at matched utility — success means meaningfully lower cumulative leakage than every static and random arm on at least MNIST, CIFAR-10, and CelebA. Second, the surrogate check: rank correlation between $I^2F$-planned and attack-in-the-loop-planned schedules on MNIST/LeNet where the loop is affordable. Third, the natural-shield check described above. Fourth, a switching-frequency sweep replicating Qian et al.'s non-monotonicity in gradient space: if defensive benefit is monotone in shuffle rate, the receding-horizon structure is unnecessary and a simple front-loaded schedule suffices. Fifth, the deception check: an active server tampering with weights to bias the client's $I^2F$ reading, with the prediction — derived from the Maximin-trap theory of Nguyen and Xu — that deterministic policies are exploited while randomized commitments are not. No reported result exists on Lacuna for any adaptive multi-round attacker measured against any dynamic gradient defense; establishing that first number, in both directions, is the point of the work.
References
- Gradient Inversion with Generative Image Prior
- Learning and Planning Under Uncertainty for Green Security
- OASIS: Offsetting Active Reconstruction Attacks in Federated Learning
- SoK: Gradient Leakage in Federated Learning
- A Receding-Horizon MDP Approach for Performance Evaluation of Moving Target Defense in Networks
- Gradient Inversion Transcript: Leveraging Robust Generative Priors to Reconstruct Training Data from Gradient Leakage
- FedInverse: Evaluating Privacy Leakage in Federated Learning
- Robust Protection of Fisheries with COmPASS
- Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective
- Concealing Sensitive Samples against Gradient Leakage in Federated Learning
- Mixed Quantization Enabled Federated Learning to Tackle Gradient Inversion Attacks
- Understanding Deep Gradient Leakage via Inversion Influence Functions
- Imitative Attacker Deception in Stackelberg Security Games