Can We Trust the Similarity Measurement in Federated Learning?

Zhilin Wang, Qin Hu, Xukai Zou, Pengfei Hu, Xiuzhen ChengIEEE Trans. IFS 2023

TL;DR

This paper reveals a critical security vulnerability in federated learning (FL) where common similarity metrics used to detect malicious models can be fooled. The authors introduce "Faker," a novel attack that exploits this vulnerability to effectively poison FL models, even with a single malicious client. They also propose a defense mechanism, SPP, to mitigate this threat.

What Problem Does This Paper Address?

Federated learning (FL) allows multiple devices to collaboratively train a shared model without sharing their raw data. A key challenge in FL is ensuring the integrity of the training process, as malicious clients might submit poisoned models to degrade the global model's performance. Many existing defenses rely on similarity metrics (like L2 norm, Euclidean distance, cosine similarity) to identify and discard outlier or malicious local models. The assumption is that poisoned models will be significantly different from benign ones. However, this paper questions whether these similarity metrics are truly robust and secure against sophisticated attackers.

Methods

The core of this paper lies in identifying and exploiting a vulnerability in similarity-based FL defenses.

  • Vulnerability Analysis: The authors demonstrate that high-dimensional local models, even if significantly different in parameter values, can exhibit similar evaluation results according to standard similarity metrics. This is because these metrics often treat the model as a whole, masking individual parameter discrepancies.
  • Faker Attack: To exploit this, they propose "Faker," an untargeted model poisoning attack. Faker aims to craft poisoned local models that are both highly similar to benign models (to pass detection) and significantly different in their parameter values (to degrade the global model). This is framed as an optimization problem where the attacker maximizes a combined objective of similarity and parameter difference.
  • Similarity of Partial Parameters (SPP) Defense: As a countermeasure, the authors propose SPP. Instead of evaluating the similarity of the entire local model, SPP randomly selects a subset of parameters for similarity calculation. This makes it harder for attackers to craft poisoned models that consistently fool the defense, as they cannot predict which parameters will be evaluated.

Datasets

The experiments were conducted using the following datasets:

  • MNIST: A dataset of handwritten digits.
  • Fashion MNIST (FMNIST): A dataset of clothing item images.
  • CIFAR-10: A dataset of small color images across 10 classes.
  • CIFAR-100, HAM10000, Tiny ImageNet, Reuters: Used for additional experiments to evaluate Faker and SPP on larger and more diverse datasets.

Key Insights and Ideas

  • The "Curse" of Similarity: Standard similarity metrics, while intuitive, can be misleading in high-dimensional FL settings. They can make significantly different models appear similar, creating a blind spot for defenses.
  • Faker's Stealth and Effectiveness: The Faker attack is particularly concerning because it's designed to be stealthy (evading detection) and highly effective, even when launched by a single malicious client. This challenges the assumption that multiple malicious clients are needed for impactful attacks.
  • Single-Round Attack Power: Faker can degrade the global model's performance with just a single attack round, making it a potent threat.
  • SPP's Targeted Defense: The SPP defense offers a promising direction by shifting focus from the entire model to partial parameters, making it more difficult for attackers to game the system.

Key Results

  • Faker Outperforms Benchmarks: Faker significantly outperforms existing model poisoning attacks (LA and MB) by 1.1-9.0X in reducing accuracy and 1.2-8.0X in saving time cost across various datasets and defenses.
  • 100% Success Rate: Faker consistently achieves a 100% attack success rate against various defenses, meaning it successfully evades detection and poisons the global model.
  • Single Client Efficacy: Faker is effective even with a single malicious client, a significant advantage over attacks that require collusion.
  • Time Efficiency: Faker is substantially more time-efficient than benchmark attacks, especially against defenses like Krum.
  • SPP Effectiveness: The proposed SPP defense demonstrates similar performance to existing defenses (like ERR) in mitigating Faker, while requiring less computational overhead.

Limitations and Future Work

  • Robustness of Similarity Metrics in Other Systems: The paper focuses on FL, but the authors suggest further theoretical and experimental study is needed for the robustness of similarity metrics in other systems.
  • Adapting Faker to Other Mechanisms: Faker's potential to undermine other similarity-based mechanisms beyond standard defenses (e.g., in fairness assessments) warrants further exploration.
  • More Efficient Solutions for Faker's Optimization: While an approximate solution for Faker's optimization problem (P0) is used and effective, more efficient and accurate solutions could be beneficial.
  • Robustness of Other Evaluation Metrics: This work is a first step in analyzing evaluation metric robustness; further research is needed for other metrics used in FL.
  • Cooperative Attack Mode: While the paper focuses on single-client attacks, it briefly explores cooperative modes, suggesting this could be a future research avenue.

Summary updated

BibTeX
@article{wang2023can,
  author = {Zhilin Wang and Qin Hu and Xukai Zou and Pengfei Hu and Xiuzhen Cheng},
  title = {{Can We Trust the Similarity Measurement in Federated Learning?}},
  journal = {IEEE Trans. Inf. Forensics Secur.},
  year = {2023},
}
Figure