FedMUA: Exploring the Vulnerabilities of Federated Learning to Malicious Unlearning Attacks

Jian Chen, Ziyuan Lin, Wanyu Lin, Wenlong Shi, Xiaoyan Yin, Di WangIEEE Trans. IFS 2025 · arXiv

TL;DR

This paper introduces FedMUA, the first known attack that exploits federated unlearning to maliciously manipulate global models, causing misclassification for specific users. It proposes a novel two-step attack (Influential Sample Identification and Malicious Unlearning Generation) and a corresponding defense mechanism, demonstrating significant attack success rates with minimal malicious requests.

What Problem Does This Paper Address?

Federated learning (FL) allows multiple clients to collaboratively train a model without sharing their raw data. A growing concern is the "right to be forgotten," leading to federated unlearning (FU) techniques that aim to remove a client's data influence from the global model without full retraining. However, existing FU research largely assumes the unlearning process is benign. This paper addresses the critical security gap: what if a malicious client exploits the unlearning process itself to manipulate the global model's predictions, specifically targeting certain users? This could have serious real-world consequences, such as unfairly assigning low credit ratings to individuals.

Methods

The core of FedMUA is a two-step attack strategy:

  1. Influential Sample Identification (ISI): This step identifies "influential" training samples from the attacker's local data that have the most significant impact on the prediction of a specific target user's data. It leverages influence functions to pinpoint these samples and prioritizes those with negative influence values (meaning they would reduce the loss for the target data if unlearned).
  2. Malicious Unlearning Generation (MUG): Once influential samples are identified, MUG crafts malicious unlearning requests. Instead of simply removing the data, it manipulates the features of these influential samples to make them appear closer to the target user's data. This aims to mislead the unlearning process into removing more information than intended, thereby creating a bias in the global model's predictions for the target user.

For defense, the authors observe that malicious unlearning requests often result in larger gradient updates. Their defense mechanism identifies these large gradients using the Interquartile Range (IQR) method and reduces their impact by scaling them down.

Datasets

The paper evaluates FedMUA on several standard datasets:

  • Purchase: A Kaggle dataset for coupon promotion strategies, clustered into 2 classes.
  • MNIST: A dataset of handwritten digits (0-9).
  • CIFAR-10: A dataset of 32x32 color images across 10 classes.
  • CIFAR-100: A dataset with 100 subcategories.
  • Credit Score: A dataset with 27 features for classifying credit scores, expanded to 97,799 samples with 34 features.

Experiments were conducted in both IID (Independent and Identically Distributed) and Non-IID settings.

Key Insights and Ideas

  • Unlearning is not inherently secure: The paper demonstrates that the process of removing data can be actively exploited for malicious purposes, a vulnerability previously overlooked.
  • Targeted manipulation is feasible: FedMUA shows that it's possible to specifically target and misclassify individual users by subtly manipulating the unlearning process.
  • Feature manipulation is stealthier than label flipping: Crafting malicious feature unlearning requests is more effective and less detectable than simply altering labels.
  • Influence functions are key to effective attacks: Identifying truly influential samples is crucial for maximizing the impact of the unlearning attack.
  • Gradient magnitude as an attack indicator: The observation that malicious unlearning requests lead to larger gradient updates provides a promising avenue for defense.

Key Results

  • FedMUA can achieve an average 80% attack success rate (ASR) with as little as 0.3% malicious unlearning requests.
  • The attack effectively induces misclassification on target samples while maintaining model utility for non-target users.
  • The proposed defense mechanism significantly reduces ASR, with an average reduction of 50% when using a specific parameter setting ($\lambda=0.1$).
  • The attack's effectiveness increases with the number of malicious clients and the magnitude of feature manipulation ($\epsilon$).
  • The attack remains effective across different FU methods (FedEraser, KNOT) and aggregation rules (FedAvg, Median, Trimmed-mean, Krum).
  • The defense mechanism shows promise, reducing ASR by up to 50% on average, though it is less effective on more complex datasets like CIFAR-10.

Limitations and Future Work

  • Defense Robustness: While the proposed defense is effective, it's noted that it's less robust on more complex datasets like CIFAR-10. The authors suggest further research into more sophisticated defense mechanisms.
  • Real-world Applicability: The paper acknowledges that the attack's effectiveness can be influenced by factors like the number of participants in FL and the complexity of the data.
  • Adaptability of Defenses: Existing defenses like FAT and FADngs showed limited effectiveness against FedMUA, highlighting the need for specialized defenses against unlearning attacks.
  • Future Work: The authors suggest exploring more realistic scenarios, such as multi-target attacks and scenarios with a larger number of clients, and developing more robust defense strategies. They also highlight the need for the FL community to pay closer attention to the security implications of federated unlearning.

Summary updated

Abstract

Recently, the practical needs of “the right to be forgotten” in federated learning gave birth to a paradigm known as federated unlearning, which enables the server to forget personal data upon the client’s removal request. Existing studies on federated unlearning have primarily focused on efficiently eliminating the influence of requested data from the client’s model without retraining from scratch, however, they have rarely doubted the reliability of the global model posed by the discrepancy between its prediction performance before and after unlearning. To bridge this gap, we take the first step by introducing a novel malicious unlearning attack dubbed FedMUA, aiming to unveil potential vulnerabilities emerging from federated learning during the unlearning process. Specifically, clients may act as attackers by crafting malicious unlearning requests to manipulate the prediction behavior of the global model. The crux of FedMUA is to mislead the global model into unlearning more information associated with the influential samples for the target sample than anticipated, thus inducing adverse effects on target samples from other clients. To achieve this, we design a novel two-step method, known as Influential Sample Identification and Malicious Unlearning Generation, to identify and subsequently generate malicious feature unlearning requests within the influential samples. By doing so, we can significantly alter the predictions pertaining to the target sample by initiating the malicious feature unlearning requests, leading to the deliberate manipulation for the user adversely. Additionally, we design a new defense mechanism that is highly resilient against malicious unlearning attacks. Extensive experiments on three realistic datasets reveal that FedMUA effectively induces misclassification on target samples and can achieve an 80% attack success rate by triggering only 0.3% malicious unlearning requests.
BibTeX
@misc{chen2025fedmua,
  author = {Jian Chen and Ziyuan Lin and Wanyu Lin and Wenlong Shi and Xiaoyan Yin and Di Wang},
  title = {{FedMUA: Exploring the Vulnerabilities of Federated Learning to Malicious Unlearning Attacks}},
  year = {2025},
  eprint = {2501.11848},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.11848},
}
Figure