Abstract
With the development of trustworthy Federated Learning (FL), the requirement of implementing right to be forgotten gives rise to the area of Federated Unlearning (FU). Comparing to machine unlearning, a major challenge of FU lies in the decentralized and privacy-preserving nature of FL, in which clients jointly train a global model without sharing their raw data, making it substantially more intricate to selectively unlearn specific information. In that regard, many efforts have been made to tackle the challenges of FU and have achieved significant progress. In this paper, we present a comprehensive survey of FU. Specially, we provide the existing algorithms, objectives, evaluation metrics, and identify some challenges of FU. By reviewing and comparing some studies, we summarize them into a taxonomy for various schemes, potential applications and future directions.
Introduction
Federated Learning (FL) has emerged as a powerful paradigm for privacy-preserving machine learning, allowing multiple clients to collaboratively train a shared global model without ever exchanging their raw data. However, the rise of stringent privacy regulations, such as the General Data Protection Regulation (GDPR) in Europe and the California Consumer Privacy Act (CCPA), has introduced the "right to be forgotten." This legal requirement mandates that individuals must be able to request the removal of their data from systems that have processed it.
In the context of FL, simply deleting raw data on the client side is insufficient. The global model may still retain "memory" of the deleted data in its weights, leaving it vulnerable to Membership Inference Attacks (MIAs) where an attacker can determine if a specific data point was used during training. Federated Unlearning (FU) addresses this critical gap by providing techniques to "scrub" specific data influence from a decentralized model without the prohibitive computational cost of retraining the entire system from scratch.
Taxonomy of Federated Unlearning
The scope of FU can be categorized by the granularity of the unlearning request. As illustrated in Figure 1, there are three primary levels:
- Sample-level Unlearning: Removing specific data points or samples. This provides precise control but can be technically challenging due to complex interdependencies within the model.
- Class-level Unlearning: Removing an entire category of data (e.g., all images of a specific object class). This allows models to adapt to changing data distributions but may lead to reduced overall performance if the removed class contributed significantly to the model's general accuracy.
- Client-level Unlearning: Removing all data associated with a particular user or client. This is the most comprehensive form of unlearning and minimizes data leakage risks but requires significant adjustments to the global model.
Beyond granularity, FU methods are classified by their technical approach:
- Training-Based: These methods involve partial retraining or "memory evaluation" to refine the model. While they offer high privacy guarantees, they are often the most computationally expensive.
- Tuning-Based: Instead of full retraining, these methods adjust the parameters of an existing model to "forget" specific contributions. For example, FedEraser utilizes historical updates to reconstruct a model version that effectively "missed" the deleted data.
- Model-Based: These modify the model architecture itself, such as using pruning or quantization (e.g., TF-IDF quantization) to eliminate the influence of specific data classes or features.
The Complexity-Efficiency Trade-off
A fundamental tension exists between a model's predictive power and its "unlearnability." Simple models, such as Linear Regression or K-Nearest Neighbors (KNN), are relatively easy to unlearn because their decision boundaries are less complex. However, they often lack the accuracy required for modern tasks.
In contrast, Deep Neural Networks (DNNs) represent data through intricate, overlapping latent structures. Surgically removing the influence of a single data point or class without affecting the rest of the model's knowledge is a significant technical hurdle. As shown in Figure 2, the most accurate models are often the most difficult to unlearn efficiently, marking this as a primary area for future research.
Benchmarking with OPENFEDERATEUNLEARNING
To standardize the evaluation of these techniques, the authors introduce OPENFEDERATEUNLEARNING, a unified benchmarking framework. It evaluates FU algorithms across standard computer vision datasets (MNIST, CIFAR-10, SVHN) and tabular datasets (Adult, Purchase) using several key metrics:
- Efficiency Gains: Compared to retraining from scratch, FedEraser reduced unlearning time by up to 4.8x on MNIST. For more complex models, TF-IDF pruning for class-level unlearning achieved speedups of 8.9x for ResNet and 7.9x for VGG models on CIFAR-10, with negligible accuracy loss.
- Privacy Protection: Successful unlearning methods effectively "blind" attackers, dropping the success rate of Membership Inference Attacks (MIA) to near-random levels (around 50%).
- Accuracy Retention: Advanced frameworks like FedME$^2$ demonstrate the ability to keep accuracy impacts under 4%, even when "forgetting" up to 75% of a client's data.
Future Directions
The field of Federated Unlearning is still in its infancy, with several critical "frontier" challenges:
- Incentive Design: Developing economic or game-theoretic mechanisms to motivate servers and clients to perform the computationally intensive work of unlearning honestly.
- Large Language Models (LLMs): Unlearning in models with billions of parameters is an open problem, as LLMs do not store data in a linear or easily erasable fashion.
- Green Federated Unlearning: Reducing the carbon footprint of privacy compliance by developing communication-efficient approaches that minimize the energy-intensive rounds required for FU.
Summary updated