Federated learning trains one shared model across many clients who never send their raw data, only model updates. Some systems now record those updates, or whole model checkpoints, on a blockchain so that training and later deletion can be audited. A blockchain keeps every record forever. Recent attacks show that the difference between a model before and after a client's data is "unlearned" (removed from the model) acts like a gradient of the deleted data, and that this difference can be inverted to rebuild the deleted images. Put the two facts together and a ledger that logs both checkpoints hands every participant a permanent, tamper-proof copy of exactly the signal needed to reconstruct the data that was supposed to disappear. The act of complying with a deletion request becomes the leak.
This note proposes changing how the unlearned model is produced, rather than hiding the posted difference with noise or encryption. Instead of pushing the model away from the forgotten data with gradient ascent, which writes a forget-specific direction into the weights, the system rewinds to an older checkpoint that the ledger already stores immutably. It then replays the intervening rounds using only the remaining clients' recorded updates, and applies a small, magnitude-bounded correction for influence older than the rewind point. The published difference then consists mostly of ordinary retained-client training drift, which entangles and buries the forgotten data's contribution. Auditors can also check the replay against ledger history without needing the forgotten data. The ledger's permanence, which is the problem in the naive design, becomes the resource that makes the safer design cheap and verifiable.
Core Claim
How invertible an unlearning action is depends on how forget-specific the posted weight difference is, and this can be controlled by a single knob: the rewind depth $k$, measured in federated rounds. We claim that for small $k$, DRAUN- and FUIA-style reconstructions from the on-chain difference fall to near-unrecognizable quality, while forgetting quality approaches exact retraining and cost stays a small multiple of one gradient-ascent pass. A corollary runs against intuition and we will test it directly. Unlearning methods that are best at protecting retained accuracy, such as null-space projection, should be the most invertible. They work by confining the update to directions that retained data does not use, and those are exactly the directions that isolate the forgotten samples.
What the Prior Work Establishes
Two papers motivate the threat. FUIA shows that $\Delta W = W^o - W^u$, the original minus the unlearned global model, is a high-fidelity proxy for the forgotten data's gradient. It inverts this difference with cosine matching and total-variation regularization, and finds that approximate unlearning by gradient ascent leaves richer traces than exact retraining. DRAUN explains why classical inversion fails on unlearning updates: the true input is no longer a stationary point of the usual matching loss. It fixes this with an algorithm-agnostic surrogate and disjoint initialization, reaching SSIM 0.6407 and PSNR 21.11 on CIFAR-10, where the prior baseline reached 0.0038. Both papers assume a centralized server that sees the difference once. Neither considers a ledger where the difference is available to everyone, indefinitely.
Two systems create the regime we study. BlockFUL (also Decentralized Federated Unlearning on Blockchain) puts federated unlearning on a dual chain. One chain is a redactable live chain that uses chameleon hashes, which are hashes with a trapdoor that lets authorized parties rewrite a block without breaking the chain (Ateniese et al.). The other is a permanent archive chain. BlockFUL favors parallel gradient ascent because it is fast: 1.3 to 2.4 seconds versus 30 to 90 seconds for retraining on CIFAR-100. By FUIA's finding, that choice is also the most invertible, and the archive chain keeps $W^o$ forever, so redacting the live chain does nothing against an adversary who reads the archive. Federated TrustChain logs LoRA adapter unlearning for GPT-2 on Hyperledger Fabric and checks forgetting by accuracy on a validation set. It never asks whether the logged adapter updates are invertible. We reuse BlockFUL's lineage structure and its observation that influence decays with depth. That observation is what justifies bounding the correction for influence older than $k$ rounds. We drop its gradient-ascent default.
Two defenses constrain the design. DCS² shows that inversion succeeds when per-sample gradients are weakly entangled, and that entangling them with the batch subspace defeats inversion (MNIST PSNR falls from 59.20 to 7.84) at modest utility cost. Our rewind-and-replay achieves entanglement by construction rather than by synthesizing decoys, and synthesized decoys could fail on-chain validity checks. UNSC projects forgetting updates into the null space of retained activations and cuts forget accuracy to 0.70% while raising retained accuracy from 92.10% to 92.77%. We treat it as the hypothesized worst case for leakage. Finally, the SoK papers (SoK: On Gradient Leakage in Federated Learning; SoK: Gradient Leakage in Federated Learning) report that multiple local steps and 2-bit quantization often neutralize inversion. This both supports our mechanism and supplies the most dangerous simple competitor.
Threat Model
The adversary is an insider: any federated participant or auditor with read access to every checkpoint and update ever recorded on chain, including archive and redacted-block scars. The adversary has unlimited offline compute, full knowledge of the unlearning algorithm, and optionally a generative image prior, as in GIAS/GIML, which reconstructs even when 99% of gradient values are removed. This adversary matters because ordinary federated clients receive every global model during training. Keeping weights off chain does not stop a client who cached $W^o$ from differencing it against a later $W^u$.
Research Program
The method has three parts. Let round $t$ be when the unlearning request arrives and $W_{t-k}$ the ledger-recorded global model $k$ rounds earlier. The unlearned model is
$$W^u = \mathrm{Replay}(W_{t-k};\ U^{\text{retain}}_{t-k+1:t}) + \eta\, g_{\text{old}},$$
where $U^{\text{retain}}$ are the remaining clients' recorded updates, recomputed on the new trajectory by those clients, and $g_{\text{old}}$ is a gradient-ascent correction on the forget client's data for influence from rounds before $t-k$. That correction's step size $\eta$ is capped by a norm budget derived from BlockFUL's depth decay.
The posted difference $W^o - W^u$ is then the forget client's last $k$ round contributions, mixed with $k$ rounds of trajectory drift from many clients, plus a small correction. By the same reasoning that makes multiple local steps blur inversion, this should entangle the forget signal.
Verification is lineage-based. A smart contract checks that $W^u$'s recorded parent is a committed checkpoint and that each replay round aggregates only updates signed by retained clients. It checks the correction only for its norm bound. No validator needs the forgotten data, unlike accuracy-threshold checks.
The analysis part defines a cheap leakage predictor: the fraction of $\Delta W$'s energy lying outside the span of the retained clients' recorded updates. We test whether this predictor tracks reconstruction quality across all unlearning methods. If it does, a committee can screen any proposed unlearning update before committing it.
Baselines
The naive reference point is BlockFUL-style parallel gradient ascent with both checkpoints on chain, attacked with DRAUN and FUIA. This is the "do nothing" ledger design. The strongest attack figure on Lacuna is DRAUN's CIFAR-10 SSIM 0.6407 and PSNR 21.11 under centralized ABL unlearning. No reported result was found on Lacuna for DRAUN or FUIA against a ledger-logged BlockFUL delta. Establishing that number is our first measurement.
The privacy upper reference is exact retraining from the round before the forget client joined. On utility, BlockFUL reports 94.71% retained accuracy for sequential retraining; we expect the lowest leakage here and know the cost is the highest. The strongest existing defense is 2-bit quantization of the posted difference, which the SoK reports blocks current attacks with under 5% accuracy loss. Alongside it we run DCS²-style concealment adapted to unlearning updates and Gaussian noise or pruning, which FUIA reports dropping accuracy from about 84% to about 63%. UNSC null-space unlearning enters as the hypothesized most-leaky method.
The key ablation isolates the replay. We compare rewind-and-replay with no old-influence correction, the correction alone with no rewind (which is plain norm-bounded gradient ascent), and the full method. This tells us whether entanglement or simply a smaller step does the work.
Fairness rules apply to every comparison. All methods use the same model, federated partition, number of clients, and forget sets. Each defense gets the same hyperparameter-search budget, tuned to maximize retained accuracy subject to forget accuracy within one point of retraining. Each attack gets the same iteration budget, both with and without a generative prior.
Beating the field means the full method reaches DRAUN SSIM below 0.15 (the level DCS² reports as concealed on CIFAR-10) and forget accuracy within one point of retraining. It must also keep retained accuracy within one point of retraining, at under a quarter of retraining's wall-clock cost. Matching quantization on privacy is not enough. We must also beat quantization on forgetting or on verifiability, or the proposal reduces to "quantize."
Why not just…?
Why not just keep weights off chain and log only hash commitments, crypto-shredding the old checkpoint as Truong et al. do for personal records? This defeats outsiders and is a real partial answer, so we include it as a configuration. It fails against our insider: every client downloaded $W^o$ during training. Shredding the key does not remove influence already inherited by later models, which is a point BlockFUL makes, and it leaves auditors no way to confirm forgetting. Experiment E1 below runs the insider attack under commitment-only logging to show the leak persists.
Why not just add differential-privacy noise to the posted difference? Both FUIA and DRAUN report severe accuracy loss under noise and pruning: about 84% to about 63% in FUIA. Noise on the difference also corrupts $W^u$ itself, which then fails forgetting or retention targets. We still run it on a common privacy-utility curve.
Why not just quantize the difference to 2 bits? This is the most serious threat, given the SoK's reported under-5% utility loss. It is untested against unlearning-specific attacks, though, and FUIA's large output-layer norm signatures for class unlearning may survive coarse quantization. We run quantization and quantization plus our method under DRAUN with and without a GIAS prior. If quantized gradient ascent matches our privacy and forgetting at lower cost, we will report that as the finding and narrow our contribution to verifiability.
Why not just retrain from scratch? It is the privacy gold standard. BlockFUL's latency numbers and TrustChain's 32,277 seconds over 999 iterations show why systems avoid it. Our rewind depth sweep directly measures how much of retraining's privacy we buy at what fraction of its cost.
Proposed Experiments
Experiment E1 establishes the ledger oracle. We simulate BlockFUL's dual chain over CIFAR-10, CIFAR-100, and FEMNIST with 100 non-IID clients, following DRAUN's setup. We then run DRAUN and FUIA from on-chain differences for sample-level and class-level forgetting, reporting SSIM, PSNR, and LPIPS. We also test whether chameleon redaction of the live chain changes anything when the archive chain or a cached $W^o$ exists.
Experiment E2 is the decisive sweep over rewind depth $k$ from 0 to 20 rounds. For each $k$ we plot reconstruction SSIM, forget accuracy, membership-inference score, retained accuracy, and wall-clock time against all baselines. The method succeeds if a single small $k$ crosses the success thresholds above on all three datasets.
Experiment E3 tests the null-space prediction. We compare reconstruction quality from UNSC-style, gradient-ascent, and rewind updates matched for forget accuracy. We also correlate the out-of-span energy predictor with SSIM across all methods; we want a rank correlation strong enough to support pre-commit screening.
Experiment E4 measures the overhead of on-chain verification. We implement the lineage-checking contract on Hyperledger Fabric and record latency and throughput up to BlockFUL's 8,000-request scale, comparing against accuracy-threshold verification in the TrustChain style. A negative result in E2 would be informative on its own terms. If no $k$ escapes inversion before cost reaches retraining's, then ledger-logged federated unlearning cannot be both cheap and private under insider threat.
References
- FUIA: Model Inversion Attack against Federated Unlearning
- DRAUN: An Optimization-Agnostic Data Reconstruction Attack on Federated Unlearning
- BlockFUL: Enabling Unlearning in Blockchained Federated Learning
- Decentralized Federated Unlearning on Blockchain
- Redactable Blockchain - or - Rewriting History in Bitcoin and Friends
- Blockchain-Based Gradient Inversion and Poisoning Defense for Federated Learning
- Concealing Sensitive Samples against Gradient Leakage in Federated Learning
- Machine Unlearning via Null Space Calibration
- SoK: On Gradient Leakage in Federated Learning
- SoK: Gradient Leakage in Federated Learning
- Gradient Inversion with Generative Image Prior
- GDPR-Compliant Personal Data Management: A Blockchain-Based Solution