Unconditionally Stable Gradient Inversion via Convexity Splitting

In federated learning, phones and hospitals train a shared model by sending only parameter updates — gradient vectors — to a server, on the theory that gradients are too abstracted to reveal raw data. That theory is wrong in practice: an honest-but-curious server can reconstruct the private training images by solving an inverse problem. It picks dummy inputs, computes the gradients they would produce, and nudges the dummies until those gradients match the intercepted ones. Every documented attack solves this with explicit iterative optimizers such as Adam: repeatedly step the dummy inputs downhill on the mismatch. Explicit iteration has a well-known failure mode inherited from numerical analysis: when the loss surface is stiff — it curves sharply in some directions and gently in others — explicit steps overshoot, oscillate, and get trapped. Numerical PDE research solved exactly this problem decades ago with implicit time stepping, where the next iterate is defined as the solution of a self-consistent variational problem rather than a forward extrapolation, giving unconditional stability: the scheme stays bounded no matter how large the step.

The connection is not an analogy but a mechanism. The regimes where gradient inversion documentedly breaks down — defended updates, denoised differential-privacy gradients, large local batches, adaptive-optimizer updates Gradient Inversion with Generative Image Prior, Gradient Inversion Attacks Beyond SGD, Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage — are precisely the regimes where the inversion loss becomes stiff and explicit solvers fail, as standard DLG demonstrably does under defense-induced non-convexity Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage. Meanwhile, the semi-implicit toolbox that defeats stiffness — convexity splitting, primal–dual inner solves, information-theoretic barriers — has never been applied to the inversion objective. This proposal does exactly that.

Core Claim

The gradient-inversion objective can be written as the difference of a cheap, convex stabilizing term and a non-convex driving term, exactly as in unconditionally stable schemes for stiff PDEs. Treating the convex part implicitly (solved in closed form in a spectral or latent space) and the driving part explicitly yields an attack optimizer that is unconditionally stable with respect to its stabilizing step size. Concretely, the attacker minimizes

$$\mathcal{J}(X) \;=\; \underbrace{\tfrac{\lambda}{2}\|L X\|_2^2}_{E_c:\ \text{convex prior, implicit}} \;+\; \underbrace{\mathcal{D}\big(\nabla_W \ell(f(X)),\ g\big)}_{\text{non-convex matching, explicit}}$$

where $X$ is the dummy batch (pixels or generator latents), $g$ is the intercepted update, $\mathcal{D}$ is a distance such as cosine similarity, and $L$ is a fixed linear operator (a smoothness filter in pixel space, or the identity in latent space) so that $E_c$ is quadratic and diagonalizable. The semi-implicit step is

$$X_{n+1} \;=\; \arg\min_X \Big\{ \tfrac{\lambda}{2}\|L X\|^2 \;+\; \tfrac{1}{2\tau}\big\|X - X_n + \tau\,\nabla_X \mathcal{D}(X_n)\big\|^2 \Big\}.$$

When $L$ is diagonalized by a Discrete Cosine Transform, this inner minimization solves per frequency in closed form — the same trick that let Bertozzi and Schönlieb run stiff fourth-order inpainting PDEs at $\Delta t \ge 1$ where explicit schemes required $\Delta t = 10^{-6}$ Unconditionally stable schemes for higher order inpainting. We claim this scheme converges in the stiff regimes where explicit Adam-based inversion diverges or stalls, and that its contraction factor — a byproduct of the split — serves as a fast vulnerability audit without running any attack.

Research Program

The program has three parts: the scheme, a conditional stability theory, and an audit application.

The scheme is built in two layers. The base layer is the semi-implicit step above, instantiated in pixel space (DCT-diagonalized $E_c$) and in latent space (where $E_c$ becomes a norm on the generator latent, following the generative-prior attacks of Jeon et al., who showed latents of a generative model beat pixel-space optimization, reaching PSNR of about 15.58 dB versus 13.78 dB for pixel-based attacks on ImageNet-scale data Gradient Inversion with Generative Image Prior). The second layer adds a Fisher-type barrier in latent space: a penalty on the norm of the score of the latent density, $\|s(X)\|^2$, which keeps every iterate inside the high-density region of the data manifold — the direct analog of the Fisher information term $\int |\nabla \log \rho|^2 \rho\, dx$ that Li et al. use as an interior barrier enforcing strict positivity of densities in Wasserstein gradient flows, and which convexified their variational steps so thoroughly that a proximal-Newton solver converged in under 50 iterations where first-order primal–dual methods needed more than 10,000 Fisher information regularization schemes for Wasserstein gradient flows. Where the driving term is too non-convex for a single explicit evaluation, we replace the inner least-squares problem with a saddle-point formulation solved by primal–dual hybrid gradient, the same family used to bypass CFL restrictions in time-implicit Hamilton–Jacobi and conservation-law solvers Primal–dual hybrid gradient algorithms for computing time-implicit Hamilton–Jacobi equations, A primal-dual approach for solving conservation laws with implicit in time approximations.

The theory is deliberately modest and honest. We will prove a conditional result: if $E_c$ is $\mu$-strongly convex and the driving term's gradient growth along the implicit solve is bounded by $\mu$, then the scheme's energy is non-increasing for any step size $\tau$ — the convexity-splitting stability argument transplanted from Bertozzi and Schönlieb Unconditionally stable schemes for higher order inpainting. This does not remove the attack's global non-convexity; it guarantees step-size robustness and monotone descent, not a global minimum. We also inherit a known theoretical wart: the closest prior work transporting semi-implicit hybrid gradients into deep learning, Kim and Seo's scheme for adversarial training, proves an $\mathcal{O}(1/K)$ rate only under $L$-smoothness and a Weak Minty condition, both unverified for ReLU networks Semi-Implicit Hybrid Gradient Methods with Application to Adversarial Robustness. Our statements will likewise be conditional on smooth surrogates, with the ReLU case validated empirically rather than claimed as a theorem.

The audit application converts a defense-side weakness into a deliverable. Guan et al.'s Inversion Influence Function ($I^2F$) audits client vulnerability over 500 times faster than running an iterative attack on ResNet152, but assumes a perfect attacker reaching the global inversion minimum and relies on a first-order Taylor expansion that degrades in non-convex regions Understanding Deep Gradient Leakage via Inversion Influence Functions. Our contraction factor needs neither assumption: it measures whether the local split admits any stable descent, computed per round and per client at a cost comparable to a handful of Jacobian–vector products.

Proposed Experiments

We evaluate on the benchmark stack used by the inversion literature: MNIST, CIFAR-10, CIFAR-100, TinyImageNet, and ImageNet at $64 \times 64$, with ResNet-family classifiers (the I²F work scales to ResNet152, BERT, and GPT-2 Understanding Deep Gradient Leakage via Inversion Influence Functions). Reconstruction quality is measured by PSNR and SSIM against the true private inputs; efficiency by iterations, gradient/Jacobian evaluations, and wall-clock to reach a target PSNR.

Three regimes define the test. First, clean gradients with growing batch size: sweep $B \in \{1, 8, 16, 32, 64\}$, since batch scaling is a documented open bottleneck, with the best reported reach being GIAS holding batch-32 quality near prior batch-4 attacks Gradient Inversion with Generative Image Prior and Wan et al. remaining effective up to $B=16$ on Adam updates Gradient Inversion Attacks Beyond SGD. Second, defended updates: per-example clipping with Gaussian noise in the Fed-CDP style Gradient-Leakage Resilient Federated Learning, Soteria-style masking, and DCS² decoy projection Concealing Sensitive Samples against Gradient Leakage in Federated Learning. Third, the denoised pipeline: pair our solver with Mjölnir's diffusion denoiser of DP-perturbed gradients Gradient Diffusion: A Perturbation-Resilient Gradient Leakage Attack, testing whether the stable outer solver exploits the denoised landscape better than Adam does, and whether the combined attack extends to non-noise defenses where Mjölnir's authors acknowledge it struggles.

Baselines

The naive reference point is undefended, no-defense inversion: an explicit Adam attack on clean gradients, which establishes the practical ceiling of leakage. For calibration, Wu et al. report that with no defense, MNIST reconstructions reach a PSNR of 59.20, collapsing to roughly 7.84 under their DCS² defense Concealing Sensitive Samples against Gradient Leakage in Federated Learning.

The strongest competitor depends on the regime, and we will run each at its own best setting. On defended updates, the strongest existing attack is GGL, the generative leakage auditor that combines a GAN latent prior with gradient-free optimization; it maintained PSNR over 5.7 on ImageNet across tested defense scenarios including Soteria and differential privacy Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage. That number — low absolute fidelity but robust across defenses — is the headline target: beating GGL means sustaining materially higher PSNR under the same defenses at equal or fewer objective evaluations. On DP-noised updates, the strongest attack is Mjölnir, which restored PSNR to 18.25 on CIFAR-100 at DP $\epsilon=1$ where the classic DLG attack collapsed to 0.112 Gradient Diffusion: A Perturbation-Resilient Gradient Leakage Attack; we compare in the Mjölnir configuration (same noise levels, same denoiser) with only the outer optimizer swapped. On clean adaptive-optimizer updates, the strongest is Wan et al.'s update-matching attack, which improved PSNR by up to 6.5 dB over baselines such as HFGI when attacking Adam-based updates Gradient Inversion Attacks Beyond SGD. And on clean gradients with generative priors, the strongest is GIAS at its reported 15.58 dB versus 13.78 dB for pixel attacks Gradient Inversion with Generative Image Prior.

Fairness is enforced by construction: every attack sees the identical intercepted gradient, network, labels, and defense configuration; each receives the same budget of objective evaluations (or wall-clock, reported both ways) and the same tuning sweep over its own hyperparameters, including step size for the explicit baselines and $\tau$ and $\lambda$ for ours. Where a generative prior is used, the same pretrained generator is shared. The key ablation runs our exact objective — same loss, same prior, same initialization — with the implicit solve replaced by a plain explicit step, isolating the semi-implicit ingredient as the only difference. A second ablation removes the Fisher barrier while keeping the split, isolating the manifold-regularization ingredient. Beating a baseline means higher PSNR/SSIM at matched budget with a success-rate advantage over at least 20 random restarts; for the audit deliverable, beating $I^2F$ means comparable rank correlation with realized attack quality at similar speed while relaxing its perfect-attacker assumption.

Why not just…?

Why not just run Adam or L-BFGS with a TV regularizer? This is the ablation, so the question is answered directly by experiment. The literature already indicates where it falls short: standard DLG fails completely under defense-induced non-convexity, which is why GGL's authors abandoned gradient descent for gradient-free search Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage, and the PDE record shows explicit schemes on stiff fourth-order problems are confined to microscopic steps ($\Delta t = 10^{-6}$) while the semi-implicit version runs at $\Delta t \ge 1$ with an order-of-magnitude speedup Unconditionally stable schemes for higher order inpainting. Still, if tuned Adam on our objective matches our scheme in every regime, the proposal's mechanism claim is wrong and we will report that — the ablation is designed to be falsifiable in exactly this way.

Why not just use generative priors or diffusion denoisers (GIAS, GGL, Mjölnir)? These are complementary, not competing, and one experiment makes this concrete: our solver is an outer optimizer that can wrap any of them. GIAS is computationally heavy — its authors note StyleGAN2-class generators consume substantial memory and fail when generator capacity cannot capture the data distribution Gradient Inversion with Generative Image Prior — and Mjölnir's authors concede it struggles against non-noise perturbations and complex architectures Gradient Diffusion: A Perturbation-Resilient Gradient Leakage Attack. The genuine threat is that latent-space priors already regularize the landscape enough that stability machinery adds nothing; the GIAS-latent ablation (GIAS's own inner optimizer versus ours on identical latents) settles this empirically, and if the prior alone suffices on clean data, the defended and denoised regimes are where we expect and will demonstrate the difference.

Why not just add differential privacy or decoy projection and call the data safe? That question inverts the attack into an audit argument: defenses must be calibrated against the strongest realistic attacker. Mjölnir already reverses $\epsilon=1$ DP noise to 18.25 dB PSNR on CIFAR-100 Gradient Diffusion: A Perturbation-Resilient Gradient Leakage Attack, so "add DP" is not a settled answer; DCS² currently reduces CIFAR-10 SSIM to 0.15 and holds 65.68% test accuracy on TinyImageNet versus Soteria's 52.06% Concealing Sensitive Samples against Gradient Leakage in Federated Learning, and our attack will test whether that margin survives a stable solver. For Fed-CDP specifically, no reported result found on Lacuna for reconstruction quality under attack, so we will generate those numbers ourselves. If our attack cannot dent DCS² or Fed-CDP, that is a positive finding for those defenses and a useful negative result for the stability hypothesis.

Why not just use first-order primal–dual methods instead of the proximal inner solves? The record within Direction 1 is blunt: on comparable convexified variational steps, first-order primal–dual needed more than 10,000 iterations where the regularized proximal-Newton scheme reached high precision in under 50 Fisher information regularization schemes for Wasserstein gradient flows. Our per-step cost is kept near first-order cost because the implicit part is quadratic and DCT-diagonalizable, so the closed-form solve is a transform plus elementwise division — this is the cost profile that made the inpainting schemes fast Unconditionally stable schemes for higher order inpainting. We will still run a PDHG variant as a sanity baseline.

Decisive Checks and Honest Limits

Three checks decide the proposal. First, the stiffness experiment: at matched objectives, sweep step size and show the explicit ablation diverges or stalls beyond a critical step while our scheme keeps monotone energy descent across the same range — the direct analog of the $\Delta t$ sweep in the inpainting paper. Second, the defended-upside experiment: exceed GGL's PSNR of over 5.7 under Soteria and DP on ImageNet at matched evaluation budgets Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage, and beat the Adam-outer variant of the Mjölnir pipeline on CIFAR-100. Third, the audit experiment: show the contraction factor ranks client vulnerability at least as accurately as $I^2F$ Understanding Deep Gradient Leakage via Inversion Influence Functions without its perfect-attacker assumption. The limits are stated up front: our stability is conditional on a smoothness surrogate that ReLU networks violate in theory, the scheme cannot guarantee escape from every non-convex basin, and on clean, undefended, small-batch gradients — where explicit attacks already work — we expect no advantage and will say so.