A Critical Evaluation of Privacy and Security Threats in Federated Learning

Muhammad Asad, Ahmed Moustafa, Chao YuSensors 2020 · DOI

TL;DR

Federated Learning (FL) promises privacy by training models locally, but this paper critically evaluates its inherent privacy and security vulnerabilities across all its phases. The authors demonstrate how malicious actors can exploit FL's initialization, local update, and aggregation stages through poisoning and inference attacks, impacting model performance and data confidentiality. They propose practical solutions, including differential privacy and cryptographic techniques, and highlight the crucial trade-off between enhanced security and model convergence.

What Problem Does This Paper Address?

The proliferation of smart devices and the Internet of Things (IoT) generates vast amounts of sensitive data. Centralized cloud servers processing this data pose significant privacy risks. Federated Learning (FL) emerged as a solution, enabling collaborative model training without direct data sharing. However, FL is not inherently immune to privacy and security threats. This paper addresses the critical need to understand and mitigate these vulnerabilities that can arise at various stages of the FL process, from model initialization to aggregation, ensuring that the promise of privacy in FL is not undermined by sophisticated attacks.

Methods

The paper adopts a critical evaluation approach, dissecting the FL process into three main phases: Initialization, Local Updates, and Model Aggregation. For each phase, they identify potential privacy and security threats, specifically focusing on inference attacks (leaking private information) and poisoning attacks (manipulating model behavior).

The authors conduct experimental evaluations to demonstrate these threats. They simulate FL scenarios using a standard dataset and introduce varying numbers of malicious clients to observe the impact on model accuracy and convergence. They also explore the effectiveness of proposed solutions, such as:

  • Differential Privacy (DP): Adding calibrated noise to data or model parameters to obscure individual contributions.
  • Cryptography Techniques (CT): Employing methods like homomorphic encryption and secure multi-party computation (SMC) to encrypt data before transmission.

The experiments compare the performance of FL with and without these privacy-enhancing techniques, analyzing the trade-offs.

Datasets

The primary dataset used for training and evaluation in this paper is the MNIST dataset. This is a publicly available and commonly used dataset for digit classification, consisting of handwritten digit images. The authors specifically mention using it for training a Convolutional Neural Network (CNN) model.

Key Insights and Ideas

  • FL is not a silver bullet for privacy: Despite its design, FL is susceptible to sophisticated attacks that can compromise user privacy and model integrity.
  • Attacks are phase-specific: Different types of attacks are more effective or relevant at different stages of the FL lifecycle (initialization, local updates, aggregation).
  • Malicious clients are a significant threat: Even a small number of malicious clients can severely degrade model performance and introduce vulnerabilities.
  • The "honest-but-curious" assumption is fragile: While FL assumes participants are honest, the paper demonstrates how curiosity can be exploited by adversaries.
  • Privacy vs. Performance Trade-off is real: Implementing stronger privacy and security measures (like DP and CT) often comes at the cost of reduced model convergence speed and accuracy.
  • Model aggregation is a critical vulnerability point: The cloud server, while intended to be trusted, can be a target for attacks or a source of privacy leakage if not properly secured.

Key Results

The paper presents several key experimental findings:

  • Initialization Phase: Increasing the number of malicious clients (from 2 to 8) significantly degraded model accuracy, with accuracy dropping to near zero when 6 or 8 malicious clients were present. This highlights the vulnerability of the initialization phase to manipulation.
  • Local Updates Phase:
    • Privacy: Using homomorphic encryption for local gradients showed improved convergence compared to general FL settings, suggesting CT can mitigate privacy risks during this phase.
  • Security: Injecting backdoor attacks had a more significant impact on model convergence when introduced earlier in the training process (fewer communication rounds). The paper recommends using a higher number of local epochs and global communication rounds to mitigate this.
  • Model Aggregation Phase: Experiments comparing "Without Secure Aggregation," "Secure Aggregation" (using DP), and "Partial Secure Aggregation" showed that while secure aggregation can protect against attacks, it incurs a performance cost. The paper notes that the cloud server aggregator might not be intelligent enough to distinguish between perturbed and non-perturbed updates, leading to similar performance in some scenarios.
  • Convergence: The paper demonstrates that higher levels of privacy (DP) and security (CT) generally lead to slower convergence. Figure 7 shows that increased privacy budget values (PB) and security parameter bits (SP) resulted in better accuracy, but the trend suggests a diminishing return or a point where further increases might not yield proportional benefits.

Limitations and Future Work

Limitations:

  • The paper acknowledges that real-world attack scenarios might differ from their simulated settings, and they cannot guarantee the exact performance drop observed in actual attacks.
  • The experimental setup for model aggregation suggests that current aggregators might not be sophisticated enough to intelligently handle perturbed updates, leading to performance issues.

Future Work:

The authors identify several promising future directions:

  • Incremental Federated Learning: Developing techniques for training on datasets that are not fixed and for clients with personalized datasets.
  • Hierarchical Federated Learning: Investigating FL tasks decomposed into hierarchies for robustness and exploring privacy/security threats in such structures.
  • Mobile Federated Learning: Addressing the challenges of training on mobile devices that are constantly moving.
  • Decentralized Federated Learning: Exploring scenarios where clients elect themselves as servers, eliminating the need for a central cloud server, and investigating associated privacy risks.
  • Adaptive Clustering: Dividing workloads among multiple clients through clustering to improve communication efficiency and exploring privacy risks within clusters.
  • Clients Heterogeneity: Accounting for diverse client capabilities, privacy requirements, and participation patterns in FL systems.

Summary updated

Abstract

With the advent of smart devices, smartphones, and smart everything, the Internet of Things (IoT) has emerged with an incredible impact on the industries and human life. The IoT consists of millions of clients that exchange massive amounts of critical data, which results in high privacy risks when processed by a centralized cloud server. Motivated by this privacy concern, a new machine learning paradigm has emerged, namely Federated Learning (FL). Specifically, FL allows for each client to train a learning model locally and performs global model aggregation at the centralized cloud server in order to avoid the direct data leakage from clients. However, despite this efficient distributed training technique, an individual's private information can still be compromised. To this end, in this paper, we investigate the privacy and security threats that can harm the whole execution process of FL. Additionally, we provide practical solutions to overcome those attacks and protect the individual's privacy. We also present experimental results in order to highlight the discussed issues and possible solutions. We expect that this work will open exciting perspectives for future research in FL.
BibTeX
@inproceedings{asad2020critical,
  author = {Muhammad Asad and Ahmed Moustafa and Chao Yu},
  title = {{A Critical Evaluation of Privacy and Security Threats in Federated Learning}},
  booktitle = {Sensors},
  year = {2020},
  doi = {10.3390/s20247182},
}
Figure