Zero-shot Federated Unlearning via Transforming from Data-Dependent to Personalized Model-Centric

Wenhan Wu, Huanghuang Liang, Jingling Yuan, Jiawei Jiang, Kanye Ye Wang, Chuang Hu, Xiaobo Zhou, Dazhao Cheng

Introduction

Federated Learning (FL) has emerged as a powerful paradigm for collaborative model training without sharing raw data. However, the increasing demand for privacy regulations, such as the GDPR's "right to be forgotten," necessitates efficient Federated Unlearning (FU). Traditional FU methods typically require retraining from scratch or access to historical data/updates, which is computationally expensive and potentially privacy-violating. This paper introduces ZeroFU, the first "zero-shot" federated unlearning framework that removes a specific client's contribution without requiring access to original training data or historical logs.

Methodology

ZeroFU shifts the unlearning paradigm from being data-dependent to a personalized model-centric approach. The framework is divided into two distinct phases: the initial learning phase and the subsequent unlearning phase.

1. The Learning Phase: Embedding the "Hook"

To enable unlearning without data, the model must be trained to distinguish between shared knowledge and client-specific nuances. ZeroFU achieves this through a Condition Module (CM) and a Class Embedding Generator (eGen). During training, the model learns to separate features into two streams: global features (shared across the federation) and personalized features (unique to each client's specific label distribution). This separation creates a "hook" that allows the system to target and remove specific client influences later.

The Learning Phase: Architecture of the Central Server and Local Clients featuring the Condition Module (CM) and Class Embedding Generator (eGen) to separate global and personalized features.

2. The Unlearning Phase: GAN-based Distillation

When a client requests to be forgotten, ZeroFU employs a Generative Adversarial Network (GAN) and Knowledge Distillation (KD). Since original data is unavailable, a Generator ($G$) creates synthetic "pseudo-data" to trigger the model's internal representations. A Teacher-Student framework is then established where the current model (on a retained client) acts as the Teacher. The Student model is trained to maintain the Teacher's global performance while simultaneously masking or obfuscating the personalized features associated with the target client.

The Unlearning Phase: GAN-based distillation process using a Teacher-Student framework to obfuscate personalized features of the forgotten client while retaining global knowledge.

Experimental Evaluation

The researchers evaluated ZeroFU on four standard benchmarks: MNIST, Fashion-MNIST, SVHN, and CIFAR-10, under non-IID (heterogeneous) data settings. They tested both pathological and practical data skews to simulate real-world federated environments.

Key Results

  • Accuracy Recovery: In highly imbalanced scenarios, ZeroFU achieved performance nearly identical to retraining from scratch, with an average accuracy difference of only 0.87% on retained data and 0.44% on forgotten data. It outperformed existing zero-shot baselines by up to 26.2%.
  • Privacy Protection: Membership Inference Attack (MIA) evaluations demonstrated that ZeroFU effectively "blinds" the model to the forgotten data, with attack recall rates comparable to models that never saw the target data.
  • Weight Similarity: Mathematical analysis using L2 Norms confirmed that ZeroFU's model weights converge significantly closer to the "gold standard" retrained model than other baseline methods.

Conclusion and Future Work

ZeroFU demonstrates that personalization can be a powerful tool for privacy, allowing for surgical unlearning without data access. While the framework requires a specific initial training protocol and involves computational overhead for GAN-based distillation, it provides a robust solution for honoring the "right to be forgotten" in privacy-sensitive federated environments. Future research may explore extending this to class-level unlearning and optimizing the efficiency of the generative process for edge devices.

Summary updated

Abstract

Federated Unlearning (FU) addresses the "right to be forgotten" in federated learning by removing specific client data's contribution without retraining from scratch. Existing FUs are data-dependent, which make the assumption that systems can access original training data or stored historical parameter updates during unlearning. However, the assumption cannot always hold in practice, as users usually request the deletion of client data and historical parameter updates due to privacy concerns or storage limitations. Therefore, it is crucial to develop a zero-shot FU method without such data access. The key challenge is how to distinguish and remove the impact of target clients without data-level information. Motivated by the idea that if we can learn client-specific personalized information from the model instead of data, FU can be model-centric and data-free, we present the first zero-shot FU framework ZeroFU. By embedding client contributions into the model during learning via condition computation, ZeroFU enables the model to possess personalized features for unlearning. The unlearning is achieved using a proposed GAN-based distillation framework that obfuscates the personalized feature of the target client. Evaluations demonstrate its effectiveness in unlearning under non-IID settings.
BibTeX
@inproceedings{wu2025zero,
  author = {Wenhan Wu and Huanghuang Liang and Jingling Yuan and Jiawei Jiang and Kanye Ye Wang and Chuang Hu and Xiaobo Zhou and Dazhao Cheng},
  title = {{Zero-shot Federated Unlearning via Transforming from Data-Dependent to Personalized Model-Centric}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/733},
}
Figure