Abstract
Federated Learning (FL) enables decentralized model training but faces a significant "triple challenge": ensuring data privacy, maintaining communication efficiency, and providing robustness against malicious actors. ScionFL is the first framework to simultaneously address these goals. By integrating secure multi-party computation (MPC) with 1-bit quantization and novel defense mechanisms, ScionFL reduces inter-server communication by up to 17x while preserving model accuracy and resilience against state-of-the-art poisoning attacks.
The Triple Challenge in Federated Learning
Standard Federated Learning allows clients to keep their raw data local, but it remains vulnerable. Privacy can be compromised as raw gradients may leak sensitive information. Efficiency is hampered by the high dimensionality of modern models, and robustness is threatened by "poisoning attacks" where malicious clients submit fake updates to corrupt the global model.
Existing solutions often address these issues in isolation. For instance, Secure Aggregation (SecAgg) protects privacy but is computationally expensive and typically incompatible with the inspection required for robustness defenses. ScionFL bridges these gaps by providing a unified framework that is private, efficient, and resilient.
System Architecture and Methods
ScionFL utilizes an outsourced Multi-Party Computation (MPC) model. In this setup, clients do not send updates to a single central server. Instead, they split their updates into secret "shares" and distribute them across a small committee of servers. As long as at least one server in the committee is honest, individual user data remains private.
Technical Innovations
- Linear Quantization Support: ScionFL works directly on quantized 1-bit inputs, supporting advanced schemes like the Randomized Hadamard Transform (HSQ) and Kashin’s representation (KSQ), which reduce the error introduced by compression.
- SepAgg & Approximations: To avoid the massive communication overhead typical of MPC, the authors introduced a "SepAgg" approximation. Instead of performing exact bit-to-arithmetic conversions for every single parameter, they use a statistical approximation that preserves the "unbiased" nature of the gradients while slashing bandwidth needs.
- ScionFL-Aura: This is a novel bipartite defense mechanism built into the MPC protocol. It uses $L_2$-norm scaling (clipping updates that are too large) and cosine-distance filtering (removing updates that point in a suspicious direction) to thwart "Min-Max" poisoning attacks.
Experimental Evaluation
The framework was evaluated using standard computer vision benchmarks, including MNIST (LeNet) and CIFAR-10 (ResNet9, VGG11), with client counts ranging from 20 to 5,000 to test scalability.
Key Results
- Massive Speedup: For the MNIST benchmark with 500 clients, ScionFL improved inter-server communication by a factor of 17.2x compared to naive MPC implementations.
- Minimal Accuracy Loss: On MNIST, the validation accuracy only dropped from 99.04% (insecure) to 98.71% (secure/quantized) after 1,000 rounds.
- Strong Defense: Against state-of-the-art "Min-Max" poisoning attacks, ScionFL-Aura limited the accuracy drop to just 7.7% for HSQ and 10.7% for KSQ, whereas such attacks can typically tank model accuracy by 20% or more.
Insights and Future Work
A key insight from this work is that mathematical precision in secure aggregation is not always necessary for machine learning. The inherent noise in ML training allows for cryptographic approximations like SepAgg to provide massive efficiency gains without compromising the final model. Furthermore, the authors discovered that 1-bit quantization actually enhances robustness by making it harder for attackers to craft subtle, malicious updates.
Future work includes extending the protocol to handle "malicious" servers (beyond the current semi-honest assumption) and providing formal theoretical proofs for the unbiased nature of SepAgg when combined with complex transforms like Kashin’s representation.
Summary updated