Abstract
Federated learning (FL) is inherently vulnerable to poisoning attacks due to its distributed nature. Malicious clients can manipulate local data or model updates to corrupt the global model, causing it to misclassify inputs. While existing Byzantine-robust defenses offer empirical protection, they lack formal security guarantees. This paper introduces FLCert, an ensemble-based federated learning framework that provides the first provable security guarantees against poisoning attacks. By partitioning clients into groups and employing a majority vote across group-specific models, FLCert ensures that a predicted label remains unchanged as long as the number of malicious clients is within a certified bound. Extensive evaluations across datasets like MNIST, CIFAR-10, and Reddit demonstrate that FLCert achieves significant certified accuracy where traditional methods fail.
Introduction
Federated Learning (FL) has emerged as a powerful paradigm for training machine learning models on decentralized data while maintaining user privacy. However, the lack of visibility into client-side training makes FL highly susceptible to poisoning attacks. In these scenarios, malicious actors submit corrupted updates to the central server to degrade model performance (untargeted attacks) or induce specific misclassifications (targeted or backdoor attacks).
Traditional defenses, often categorized as Byzantine-robust FL, attempt to filter out malicious updates using statistical methods like Krum or Trimmed-mean. However, these defenses are often reactive, leading to a "cat-and-mouse" cycle where attackers develop new strategies to bypass empirical filters. FLCert breaks this cycle by providing a framework with provable security guarantees, ensuring robustness regardless of the specific attack strategy employed by a bounded number of malicious clients.
The FLCert Framework
FLCert shifts the FL paradigm from training a single global model to an ensemble-based approach. The core methodology relies on a "Divide and Conquer" strategy consisting of three primary phases:
- Client Grouping: The total pool of $n$ clients is divided into $N$ distinct groups. The paper proposes two specific grouping strategies:
- FLCert-D (Deterministic): Clients are assigned to $N$ disjoint groups using a deterministic hash of their unique IDs. This ensures each client influences exactly one group model.
- FLCert-P (Probabilistic): Clients are randomly sampled into groups, providing a probabilistic guarantee that is useful in dynamic environments.
- Group Training: For each group, a separate global model is trained using standard FL algorithms (e.g., FedAvg). Because each group operates independently, a malicious client can only influence the models of the groups to which it belongs.
- Ensemble Inference: During the prediction phase, the server queries all $N$ group models. The final output is determined by a majority vote among these models.
The Certified Security Level ($m^*$)
The primary innovation of FLCert is the derivation of the certified security level ($m^*$). This value represents the maximum number of malicious clients the system can tolerate while guaranteeing that the ensemble's majority vote remains unchanged. The researchers proved that this security level is tight, meaning it is the maximum possible guarantee achievable for this ensemble architecture without additional assumptions about the underlying training process.
Experimental Evaluation
The researchers evaluated FLCert across five diverse datasets to test its robustness under varying degrees of data heterogeneity (Non-IIDness):
- MNIST-0.1 & MNIST-0.5: Handwritten digit recognition with different levels of data distribution skew.
- CIFAR-10: A standard benchmark for image classification.
- HAR (Human Activity Recognition): Sensor data from mobile users.
- Reddit: A large-scale language modeling task involving 80,000 users.
Key Findings
- Superior Robustness: On the MNIST dataset with 1,000 total clients and $N=500$ groups, FLCert-D maintained a certified accuracy of 81% even in the presence of 100 malicious clients. In stark contrast, standard robust methods like Krum or FedAvg provide 0% certified accuracy, as they can be completely compromised by a single sufficiently clever malicious update.
- Efficiency and Scalability: Although training multiple models sounds computationally expensive, each group model is trained on a significantly smaller subset of data. This often leads to faster convergence. The authors noted that the total communication overhead for individual clients remains comparable to standard FL.
- The Accuracy-Security Trade-off: The framework reveals a fundamental trade-off: increasing the number of groups ($N$) enhances the security bound ($m^*$) but may reduce the accuracy of individual group models due to smaller local data pools.
Conclusion and Future Work
FLCert represents a significant milestone in secure federated learning by moving beyond empirical defenses toward mathematical certainty. By isolating client influence through grouping and leveraging the stability of majority voting, it provides a robust shield against both known and unknown poisoning attacks.
Future research directions include optimizing the storage and inference overhead associated with maintaining multiple global models and exploring how prior knowledge of specific learning tasks can be used to further tighten security bounds. Additionally, applying FLCert to more complex architectures and extremely large-scale datasets remains a promising area for deployment in real-world privacy-preserving systems.
Summary updated