Federated learning trains a shared model across many devices. Each device sends updates, not raw data, to a central server. When a user leaves and asks for their data to be erased, "federated unlearning" methods remove that user's influence without retraining from scratch. The most communication-frugal of these methods, Subspace based Federated Unlearning (SFU), does it in a single round. The departing client runs gradient ascent, meaning it pushes the model to fit its own data worse. The server then removes the parts of that push which overlap with what the remaining clients need, and applies the result once. We argue this design has a hidden cost. The deletion is packed into one message, and the projection strips out the generic directions shared with other clients, so what survives is the part of the gradient most specific to the deleted examples. Gradient inversion is the attack that reconstructs training inputs by searching for inputs whose gradient matches an observed update. The data a user asked to delete may be easiest to reconstruct from exactly this kind of message.
The proposal has two parts. First, we measure whether and when projected single-round unlearning updates are more invertible than ordinary updates. We study how that invertibility moves with the method's sensitive knobs: the SVD energy-coverage threshold $\epsilon$ that sets how much of the retained subspace is removed, and the ascent step size and number of steps. Second, we propose a different release rule, outcome-targeted release. The client never ships its raw ascent gradient. It specifies a forgetting target instead of a step size, projects locally, and among updates that meet the target it sends the one least aligned with its own raw gradient, quantized with keys kept on the device. The rule attacks two problems together. Replacing a step size with an outcome target addresses the hyperparameter brittleness documented across unlearning. Choosing a decorrelated update that is quantized with local keys addresses leakage, and it keeps the one-round, low-byte communication profile.
Core Claim
We make three claims. The first is that orthogonal projection onto the complement of the retained clients' gradient subspace makes the released unlearning update more invertible than the unprojected ascent gradient at matched norm, and increasingly so as $\epsilon$ grows. The second is that collapsing unlearning into one round removes the dilution that multi-round schemes get from averaging many updates. The third is that choosing the released update by a forgetting outcome, rather than by a step size, lets one pick updates that forget equally well while carrying far less reconstructable signal. The first two are empirical claims about existing methods. The third is the method, and it stands or falls against simple defenses bolted onto SFU.
Relation to Prior Work
Three papers motivate the leakage side. DRAUN shows that federated unlearning updates are an exploitable leakage channel. Its attack is algorithm-agnostic because it simulates both gradient-difference and gradient-ascent paths. On CIFAR-10 it reaches SSIM 0.6407 and PSNR 21.11 against ABL unlearning, where an earlier attack scores SSIM 0.0038. DRAUN covers optimization-based unlearning with full-precision updates. It does not study subspace-projected, single-round, or quantized updates.
Verifiably Forgotten? (IGF) reconstructs deleted images from the gradient differences emitted as proof of unlearning. It uses SVD to reduce the gradient and a learned CNN to decode it, reporting 88% lower MSE than a prior attack. That use of SVD to sharpen inversion is what makes us suspect SVD-based projection in unlearning is a leakage amplifier rather than a shield. Neither paper varies unlearning hyperparameters. We make those hyperparameters the independent variable.
Three papers supply methods. SFU is the one-round mechanism under study. It reports only about a 1-point clean-accuracy drop on MNIST, against about 47 points for the UPGA baseline, but performance hinges on $\epsilon$, with a good range of 0.90–0.95, and it needs high client participation. CENSOR defends ordinary training by sampling candidate updates orthogonal to the true gradient and selecting low-loss ones with a cold posterior. It keeps about 86% CIFAR-10 accuracy. We reuse its sampling idea, but add a constraint it never faces: the update must achieve a stated amount of forgetting while staying out of the retained subspace. Mixed Quantization cuts DLG reconstruction PSNR from 55.3 to 8.1 with local-key layer-wise 8/16-bit quantization and a 4× smaller payload. It was tested only on LeNet-5 training. We apply it to deletion messages, where FedOSD and NegMerge suggest small perturbations can tip unlearning into incomplete forgetting or utility collapse.
Two lines of work constrain the design. The first concerns step sizes. Boosting Alignment for Post-Unlearning Text-to-Image Generative Models shows unlearning weights must be recalibrated per target. Unilogit avoids step-size sweeps by specifying a target output distribution (uniform uncertainty) rather than an ascent strength. We borrow that "specify the outcome, not the push" principle for a federated, privacy-sensitive release. The second concerns bounded objectives and reverting. FedOSD shows bounded losses and post-training projection prevent the model drifting back to its pre-unlearning state. FedRazor reports CIFAR-10 ASR 0.026 and retained accuracy 0.659 with a two-stage trimming scheme. Both use multiple rounds and neither checks inversion, so both serve as multi-round comparison points rather than competitors on leakage.
Research Program
The measurement study reproduces SFU and FedOSD on MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100 and FEMNIST, whose natural per-writer split makes client unlearning realistic. It records the exact messages each party sends. We run DRAUN, adapted so the attacker knows the retained basis and inverts the projected update, and IGF on those messages. Reconstruction quality is scored by SSIM, PSNR and LPIPS on the forgotten client's images.
We sweep $\epsilon$, ascent learning rate and step count jointly. We also compare projected against unprojected updates rescaled to equal norm, so any gap in invertibility is attributable to direction rather than magnitude. Two threat points matter. In SFU the server receives the raw ascent gradient before projecting, so an honest-but-curious server sees everything. After the update is applied, the global-model difference is broadcast, so every other client sees the projected update.
Outcome-targeted release changes where computation happens. The server sends the target client a compact retained basis $U$ with $k$ columns. We study sending it quantized, and restricting it to the final layers to bound bytes. The client computes its raw ascent gradient $g_f$ locally, draws candidate updates $\Delta$ from the complement of $U$ in the manner of CENSOR, and releases
$$\Delta^\star = \arg\min_{\Delta \perp U} \cos^2(\Delta, g_f) \quad \text{s.t.} \quad \mathcal{F}(\theta + \Delta) \le \tau,$$
where $\theta$ is the current global model and $\mathcal{F}$ is a forgetting score computed on the client's own deleted data. That score is forget-set accuracy or its distance from a uniform prediction, and $\tau$ is the target, for example chance accuracy on forgotten classes. The client then quantizes $\Delta^\star$ with local keys. Orthogonality to $U$ does the utility protection. The forgetting constraint replaces the step size. The alignment penalty and local-key quantization do the privacy work.
One cost is explicit. Sending $U$ to the departing client exposes a summary of retained clients' gradients to that client, so we also run inversion attacks on $U$ itself and report that leakage honestly.
Baselines
The do-nothing reference is the pre-unlearning model, together with retraining from scratch without the departing client. These fix the ceiling on forgetting and utility and the zero-leakage floor, since retraining sends no deletion message. They also fix the communication cost that single-round methods are trying to avoid.
The strongest competitor on the precise question, one-round unlearning with minimal utility loss, is SFU itself. We run it with $\epsilon$ tuned inside its reported 0.90–0.95 range, using the same client counts, participation rates, local epochs, and a hyperparameter grid of the same size as our method gets for $\tau$ and candidate count. The only numbers reported for it in our file are the MNIST figures above. There is no reported result on Lacuna for SFU on CIFAR-10 or CIFAR-100, so we produce those ourselves.
The strongest bolt-on defenses are SFU with Mixed Quantization, SFU with Gaussian noise at several scales, and SFU with IGF's Orthogonal Obfuscation. Each receives the same tuning budget. As a multi-round reference, FedOSD is run with its reported configuration. Its reported retained accuracy of 75–88% and near-zero ASR set the utility bar, and we count its extra rounds as bytes. FedRazor's CIFAR-10 figures (ASR 0.026, retained accuracy 0.659) serve as a second multi-round reference.
Beating the field means sitting strictly on a better frontier. Our method must match SFU's forgetting (ASR) and retained accuracy within noise across seeds. It must cut DRAUN SSIM on CIFAR-10 well below the 0.6407 DRAUN reports against undefended unlearning. It must also beat quantized SFU on reconstruction at equal bytes.
Three ablations isolate the ingredients: outcome targeting with no alignment penalty, the alignment penalty with a fixed step size, and client-side projection with no quantization. If the alignment penalty contributes nothing beyond quantization, the method reduces to "quantize SFU," and we will report that.
Why not just…?
Why not just quantize the SFU update with local keys? This is the most serious alternative. Mixed Quantization's PSNR drop from 55.3 to 8.1 was measured on LeNet-5 training gradients, not on a single concentrated deletion message. It is unknown whether the attacker's search over hidden scales stays hard when the message is projected and low in effective rank. It is also unknown whether 8-bit rounding disturbs the delicate forgetting balance FedOSD and NegMerge describe. The quantized-SFU arm, and the "projection plus quantization, no alignment penalty" ablation, settle this directly. If quantization alone closes the gap, we will say so.
Why not just add differential-privacy noise? DRAUN reports that Gaussian noise weakens its reconstructions only with a massive drop in global accuracy. Mixed Quantization reports DP baselines at roughly 78–85% on MNIST, against 96.67% for its quantization defense. In a one-shot deletion the noise must also not undo the forgetting, which adds a second failure mode. We still include noise sweeps so the frontier is shown rather than assumed.
Why not just use IGF's Orthogonal Obfuscation? It was designed for proofs. It preserves an update's norm while scrambling its direction, which suffices when the gradient difference is only checked. In SFU the update is applied to the model, and direction is what does the forgetting. We expect obfuscation to break unlearning, and the obfuscation arm measures by how much.
Why not just use a multi-round method like FedOSD, or stability-based exact unlearning like FATS? Multi-round averaging dilutes the broadcast delta. It does not protect against a server that sees individual client uploads, and it spends the rounds single-shot unlearning exists to save. FATS avoids unlearning messages in most cases, but pays an $O(1/\sqrt{\rho_S M N})$ accuracy cost during base training, where $\rho_S$, $M$ and $N$ are the sampling and data-size quantities in its bound. It is evaluated only against membership inference, with accuracy near 50%, not against reconstruction. We report per-round leakage for FedOSD to test whether concentration really is the mechanism.
Decisive Checks
The first claim fails if, at matched norm, projected updates are no more invertible than raw ascent gradients across $\epsilon$. In that case the concentration story is wrong and the program reduces to a defense benchmark.
The method fails if outcome-targeted release cannot reach SFU's forgetting without extra rounds. It also fails if its reconstruction advantage vanishes against quantized SFU at equal bytes.
The hyperparameter claim is tested by sweeping $\tau$ and SFU's step size over comparable ranges and comparing the spread of forgetting and retained accuracy. Outcome targeting should give markedly lower variance across forget targets and datasets.
Finally, attacks on the broadcast basis $U$ quantify the leakage we introduce toward retained clients. A net win requires the total reconstruction exposure across all parties to fall.
References
- Subspace based Federated Unlearning
- DRAUN: An Optimization-Agnostic Data Reconstruction Attack on Federated Unlearning
- Verifiably Forgotten? Gradient Differences Still Enable Data Reconstruction in Federated Unlearning
- CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling
- Mixed Quantization Enabled Federated Learning to Tackle Gradient Inversion Attacks
- Federated Unlearning with Gradient Descent and Conflict Mitigation
- NegMerge: Consensual Weight Negation for Strong Machine Unlearning
- Boosting Alignment for Post-Unlearning Text-to-Image Generative Models
- Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation
- FedRazor: Two-Stage Federated Unlearning via Representation Divergence and Gradient Conflict Trimming
- Communication Efficient and Provable Federated Unlearning