Introduction
Federated Learning (FL) has emerged as a powerful paradigm for collaborative model training without sharing raw data. However, the increasing demand for privacy regulations, such as the GDPR's "right to be forgotten," necessitates efficient Federated Unlearning (FU). Traditional FU methods typically require retraining from scratch or access to historical data/updates, which is computationally expensive and potentially privacy-violating. This paper introduces ZeroFU, the first "zero-shot" federated unlearning framework that removes a specific client's contribution without requiring access to original training data or historical logs.
Methodology
ZeroFU shifts the unlearning paradigm from being data-dependent to a personalized model-centric approach. The framework is divided into two distinct phases: the initial learning phase and the subsequent unlearning phase.
1. The Learning Phase: Embedding the "Hook"
To enable unlearning without data, the model must be trained to distinguish between shared knowledge and client-specific nuances. ZeroFU achieves this through a Condition Module (CM) and a Class Embedding Generator (eGen). During training, the model learns to separate features into two streams: global features (shared across the federation) and personalized features (unique to each client's specific label distribution). This separation creates a "hook" that allows the system to target and remove specific client influences later.
2. The Unlearning Phase: GAN-based Distillation
When a client requests to be forgotten, ZeroFU employs a Generative Adversarial Network (GAN) and Knowledge Distillation (KD). Since original data is unavailable, a Generator ($G$) creates synthetic "pseudo-data" to trigger the model's internal representations. A Teacher-Student framework is then established where the current model (on a retained client) acts as the Teacher. The Student model is trained to maintain the Teacher's global performance while simultaneously masking or obfuscating the personalized features associated with the target client.
Experimental Evaluation
The researchers evaluated ZeroFU on four standard benchmarks: MNIST, Fashion-MNIST, SVHN, and CIFAR-10, under non-IID (heterogeneous) data settings. They tested both pathological and practical data skews to simulate real-world federated environments.
Key Results
- Accuracy Recovery: In highly imbalanced scenarios, ZeroFU achieved performance nearly identical to retraining from scratch, with an average accuracy difference of only 0.87% on retained data and 0.44% on forgotten data. It outperformed existing zero-shot baselines by up to 26.2%.
- Privacy Protection: Membership Inference Attack (MIA) evaluations demonstrated that ZeroFU effectively "blinds" the model to the forgotten data, with attack recall rates comparable to models that never saw the target data.
- Weight Similarity: Mathematical analysis using L2 Norms confirmed that ZeroFU's model weights converge significantly closer to the "gold standard" retrained model than other baseline methods.
Conclusion and Future Work
ZeroFU demonstrates that personalization can be a powerful tool for privacy, allowing for surgical unlearning without data access. While the framework requires a specific initial training protocol and involves computational overhead for GAN-based distillation, it provides a robust solution for honoring the "right to be forgotten" in privacy-sensitive federated environments. Future research may explore extending this to class-level unlearning and optimizing the efficiency of the generative process for edge devices.
Summary updated