Introduction
Federated Learning (FL) enables multiple clients to collaboratively train a global model while keeping their raw data localized. However, the decentralized nature of FL introduces significant security risks, particularly from Byzantine attacks. Malicious clients can submit poisoned model updates designed to degrade model performance or inject backdoors. Traditional defenses often struggle in non-IID (Independent and Identically Distributed) settings where benign data diversity can be mistaken for malicious activity.
Existing solutions generally fall into two categories: geometric defenses and validation-based defenses. Geometric defenses, such as Krum or Trimmed Mean, assume malicious updates are outliers in a high-dimensional space. However, sophisticated attackers can craft updates that are geometrically similar to benign ones. Validation-based defenses, like FLTrust, require a clean, server-side dataset to score updates, which compromises the privacy-centric goals of FL and is often impractical.
Methodology: The FedInv Framework
FedInv addresses these challenges by shifting the focus from the geometry of model updates to the underlying data distribution they represent. The framework operates through a four-stage process:
- Privacy-Respecting Model Inversion: The server reconstructs a "dummy dataset" from each client's model update using a technique inspired by Deep Leakage from Gradients (DLG). To preserve privacy, FedInv optimizes only input features and employs random initializations, ensuring the server captures statistical distributions rather than raw private data.
- Wasserstein Distance Scoring: The server computes the Wasserstein distance between pairs of dummy datasets to quantify distributional differences.
- 2-Median Clustering: FedInv utilizes 2-Median clustering to identify the majority group of benign clients, filtering out updates associated with outlier distributions.
- Signal Amplification: A scaling factor $s^{(t)}$ is introduced to prevent gradient vanishing in later training rounds, ensuring the inversion process remains effective throughout the training lifecycle.
Experimental Evaluation
The authors evaluated FedInv on three standard benchmarks: MNIST (digit classification), Fashion-MNIST (clothing classification), and HAR (Human Activity Recognition). These tests included various non-IID configurations to simulate realistic, heterogeneous data environments.
Key Results
FedInv demonstrated exceptional resilience against a wide array of attacks, including Label Flipping, Gaussian noise, and sophisticated backdoor attacks. Notably, in highly non-IID settings, FedInv maintained high accuracy where traditional baselines failed.
Against the Badnet attack on MNIST, FedInv kept the Attack Success Rate (ASR) near 0.07%, whereas baselines like LFR and FedAvg saw ASRs exceeding 90%. Furthermore, FedInv achieved performance levels comparable to validation-based methods like FLTrust without requiring any server-side public data.
Discussion and Conclusion
The core insight of FedInv is that while model gradients can be easily manipulated, it is significantly harder for an attacker to spoof a complex data distribution. By reconstructing dummy datasets, FedInv leverages the collective consensus of benign clients to define "normal" behavior.
Despite its effectiveness, FedInv faces challenges regarding computational overhead, as model inversion is resource-intensive. Additionally, like most Byzantine-robust systems, it relies on the assumption that the majority of clients are benign. Future research may focus on optimizing the inversion process for large-scale deployments and exploring formal differential privacy guarantees for the reconstruction mechanism.
Summary updated