ToFU: Transforming How Federated Learning Systems Forget User Data.

Van-Tuan Tran, Hong-Hanh Nguyen-Le, Quoc-Viet Pham

Abstract

Federated Learning (FL) enables collaborative model training across distributed devices while keeping data localized. However, deep neural networks often unintentionally memorize specific training instances, posing a significant privacy risk when users exercise their "right to be forgotten." Traditional Federated Unlearning (FU) methods are typically post-hoc and computationally expensive. We introduce ToFU (Transformation-guided Federated Unlearning), a paradigm shift that integrates unlearning capabilities directly into the training process. By applying sample-dependent transformations and progressive training strategies, ToFU proactively limits memorization. Our results demonstrate that ToFU significantly reduces unlearning time and enhances privacy while maintaining or even improving model utility across standard benchmarks.

Introduction

Deep neural networks are notoriously "leaky," often memorizing specific training samples rather than solely learning generalizable patterns. In Federated Learning environments, where models are trained on sensitive data from phones or medical servers, this memorization creates a substantial privacy bottleneck. When a user requests data deletion, the system must remove that user's influence from the global model without access to the original raw data.

Current FU methods usually attempt to "scrub" the model after training is complete. This post-hoc approach is difficult because user knowledge is deeply embedded in the model weights. ToFU addresses this by moving from a "fix-it-later" strategy to a "design-to-forget" training methodology, ensuring the model is inherently easier to unlearn from the start.

Methodology

ToFU incorporates unlearning capabilities into the collaborative training process through three core strategies:

  1. Sample-Dependent Transformation: ToFU monitors the loss for each sample. Samples with very low loss—indicating potential memorization—are subjected to higher-intensity transformations (e.g., masking or augmentations). This forces the model to focus on invariant features rather than specific pixel arrangements.
  2. Progressive Training: The intensity of these transformations increases linearly across communication rounds. This allows the model to learn basic features early on before being challenged with increasingly complex, transformed data.
  3. Transformation-Invariant Regularizer: A loss term based on Kullback–Leibler (KL) divergence encourages the model's latent representations of transformed images to remain close to their original counterparts, ensuring the model learns the underlying object identity.
Impact of the number of transformations on overall unlearning performance.

As shown in the figure above, there is a strong positive correlation between the number of transformations applied during training and the resulting unlearning performance, validating the core premise of the transformation-guided approach.

Experimental Evaluation

We evaluated ToFU using standard image classification benchmarks (CIFAR-10 and CIFAR-100) and the MUFAC (Machine Unlearning Facial Classification) dataset, which focuses on sensitive facial recognition tasks.

Key Results

  • Efficiency: ToFU reduces unlearning time by 24% to 28% compared to post-hoc baselines. Because the model is trained to be "forget-friendly," the final unlearning step is reduced to a lightweight fine-tuning operation.
  • Privacy: The method significantly lowers the success rate of Membership Inference Attacks (MIA). ToFU achieves a level of distributional similarity to "exact unlearning" (retraining from scratch) that surpasses existing methods.
  • Utility: Unlike many privacy-preserving techniques that degrade performance, ToFU maintains high accuracy. On CIFAR-10, ToFU achieved an overall performance score of 0.6712, outperforming the best baseline (0.6565).
Validation accuracy of ToFU vs. Naive training on CIFAR-10 and CIFAR-100.

As illustrated in the validation curves, ToFU consistently achieves higher accuracy than naive federated learning throughout the training process, demonstrating that its privacy-focused transformations do not come at the cost of model utility.

Discussion and Conclusion

ToFU demonstrates that the primary bottleneck in federated unlearning is not the unlearning algorithm itself, but the initial training process. By proactively limiting memorization, we make the "scrubbing" process nearly trivial. While the initial training phase involves slightly more local computation due to transformations and KL-divergence calculations, this is a worthwhile trade-off for the substantial gains in privacy and unlearning efficiency.

Future work will focus on refining transformation strategies for highly specialized domains like facial recognition, where we observed a marginal decrease in accuracy on the MUFAC dataset. Additionally, automating the selection of transformation families via methods like RandAugment could further streamline the ToFU pipeline.

Summary updated

Abstract

Neural networks unintentionally memorize training data, creating privacy risks in federated learning (FL) systems, such as inference and reconstruction attacks on sensitive data. To mitigate these risks and to comply with privacy regulations, Federated Unlearning (FU) has been introduced to enable participants in FL systems to remove their data's influence from the global model. However, current FU methods primarily act post-hoc, struggling to efficiently erase information deeply memorized by neural networks. We argue that effective unlearning necessitates a paradigm shift: designing FL systems inherently amenable to forgetting. To this end, we propose a learning-to-unlearn Transformation-guided Federated Unlearning (ToFU) framework that incorporates transformations during the learning process to reduce memorization of specific instances. Our theoretical analysis reveals how transformation composition provably bounds instance-specific information, directly simplifying subsequent unlearning. Crucially, ToFU can work as a plug-and-play framework that improves the performance of existing FU methods. Experiments on CIFAR-10, CIFAR-100, and the MUFAC benchmark show that ToFU outperforms existing FU baselines, enhances performance when integrated with current methods, and reduces unlearning time.
BibTeX
@misc{tran2025tofu,
  author = {Van-Tuan Tran and Hong-Hanh Nguyen-Le and Quoc-Viet Pham},
  title = {{ToFU: Transforming How Federated Learning Systems Forget User Data.}},
  year = {2025},
  eprint = {2509.15861},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.15861},
}
Figure