FedInv: Byzantine-Robust Federated Learning by Inversing Local Model Updates

Bo Zhao, Peng Sun, Tao Wang, Keyu JiangAAAI 2022 · DOI

Introduction

Federated Learning (FL) enables multiple clients to collaboratively train a global model while keeping their raw data localized. However, the decentralized nature of FL introduces significant security risks, particularly from Byzantine attacks. Malicious clients can submit poisoned model updates designed to degrade model performance or inject backdoors. Traditional defenses often struggle in non-IID (Independent and Identically Distributed) settings where benign data diversity can be mistaken for malicious activity.

Existing solutions generally fall into two categories: geometric defenses and validation-based defenses. Geometric defenses, such as Krum or Trimmed Mean, assume malicious updates are outliers in a high-dimensional space. However, sophisticated attackers can craft updates that are geometrically similar to benign ones. Validation-based defenses, like FLTrust, require a clean, server-side dataset to score updates, which compromises the privacy-centric goals of FL and is often impractical.

Methodology: The FedInv Framework

FedInv addresses these challenges by shifting the focus from the geometry of model updates to the underlying data distribution they represent. The framework operates through a four-stage process:

  1. Privacy-Respecting Model Inversion: The server reconstructs a "dummy dataset" from each client's model update using a technique inspired by Deep Leakage from Gradients (DLG). To preserve privacy, FedInv optimizes only input features and employs random initializations, ensuring the server captures statistical distributions rather than raw private data.
  2. Wasserstein Distance Scoring: The server computes the Wasserstein distance between pairs of dummy datasets to quantify distributional differences.
  3. 2-Median Clustering: FedInv utilizes 2-Median clustering to identify the majority group of benign clients, filtering out updates associated with outlier distributions.
  4. Signal Amplification: A scaling factor $s^{(t)}$ is introduced to prevent gradient vanishing in later training rounds, ensuring the inversion process remains effective throughout the training lifecycle.
The FedInv framework overview, showing the transition from local updates to dummy dataset reconstruction and final model aggregation.

Experimental Evaluation

The authors evaluated FedInv on three standard benchmarks: MNIST (digit classification), Fashion-MNIST (clothing classification), and HAR (Human Activity Recognition). These tests included various non-IID configurations to simulate realistic, heterogeneous data environments.

Key Results

FedInv demonstrated exceptional resilience against a wide array of attacks, including Label Flipping, Gaussian noise, and sophisticated backdoor attacks. Notably, in highly non-IID settings, FedInv maintained high accuracy where traditional baselines failed.

Attack Success Rate (ASR) comparison on MNIST and Fashion-MNIST. FedInv consistently maintains a near-zero ASR across varying backdoor sample percentages.

Against the Badnet attack on MNIST, FedInv kept the Attack Success Rate (ASR) near 0.07%, whereas baselines like LFR and FedAvg saw ASRs exceeding 90%. Furthermore, FedInv achieved performance levels comparable to validation-based methods like FLTrust without requiring any server-side public data.

Discussion and Conclusion

The core insight of FedInv is that while model gradients can be easily manipulated, it is significantly harder for an attacker to spoof a complex data distribution. By reconstructing dummy datasets, FedInv leverages the collective consensus of benign clients to define "normal" behavior.

Despite its effectiveness, FedInv faces challenges regarding computational overhead, as model inversion is resource-intensive. Additionally, like most Byzantine-robust systems, it relies on the assumption that the majority of clients are benign. Future research may focus on optimizing the inversion process for large-scale deployments and exploring formal differential privacy guarantees for the reconstruction mechanism.

Summary updated

BibTeX
@inproceedings{zhao2022fedinv,
  author = {Bo Zhao and Peng Sun and Tao Wang and Keyu Jiang},
  title = {{FedInv: Byzantine-Robust Federated Learning by Inversing Local Model Updates}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i8.20903},
}
Figure