ScionFL: Secure Quantized Aggregation for Federated Learning

Yaniv Ben-Itzhak, Helen Möllering, Benny Pinkas, Thomas Schneider, Ajith Suresh, Oleksandr Tkachenko, Shay Vargaftik, Christian Weinert, Hossein Yalame, Avishay Yanai

Abstract

Federated Learning (FL) enables decentralized model training but faces a significant "triple challenge": ensuring data privacy, maintaining communication efficiency, and providing robustness against malicious actors. ScionFL is the first framework to simultaneously address these goals. By integrating secure multi-party computation (MPC) with 1-bit quantization and novel defense mechanisms, ScionFL reduces inter-server communication by up to 17x while preserving model accuracy and resilience against state-of-the-art poisoning attacks.

The Triple Challenge in Federated Learning

Standard Federated Learning allows clients to keep their raw data local, but it remains vulnerable. Privacy can be compromised as raw gradients may leak sensitive information. Efficiency is hampered by the high dimensionality of modern models, and robustness is threatened by "poisoning attacks" where malicious clients submit fake updates to corrupt the global model.

Comparison of ScionFL with existing FL categories across privacy, quantization, and robustness metrics.

Existing solutions often address these issues in isolation. For instance, Secure Aggregation (SecAgg) protects privacy but is computationally expensive and typically incompatible with the inspection required for robustness defenses. ScionFL bridges these gaps by providing a unified framework that is private, efficient, and resilient.

System Architecture and Methods

ScionFL utilizes an outsourced Multi-Party Computation (MPC) model. In this setup, clients do not send updates to a single central server. Instead, they split their updates into secret "shares" and distribute them across a small committee of servers. As long as at least one server in the committee is honest, individual user data remains private.

The outsourced MPC model used in ScionFL for secure aggregation.

Technical Innovations

  • Linear Quantization Support: ScionFL works directly on quantized 1-bit inputs, supporting advanced schemes like the Randomized Hadamard Transform (HSQ) and Kashin’s representation (KSQ), which reduce the error introduced by compression.
  • SepAgg & Approximations: To avoid the massive communication overhead typical of MPC, the authors introduced a "SepAgg" approximation. Instead of performing exact bit-to-arithmetic conversions for every single parameter, they use a statistical approximation that preserves the "unbiased" nature of the gradients while slashing bandwidth needs.
  • ScionFL-Aura: This is a novel bipartite defense mechanism built into the MPC protocol. It uses $L_2$-norm scaling (clipping updates that are too large) and cosine-distance filtering (removing updates that point in a suspicious direction) to thwart "Min-Max" poisoning attacks.

Experimental Evaluation

The framework was evaluated using standard computer vision benchmarks, including MNIST (LeNet) and CIFAR-10 (ResNet9, VGG11), with client counts ranging from 20 to 5,000 to test scalability.

Key Results

  • Massive Speedup: For the MNIST benchmark with 500 clients, ScionFL improved inter-server communication by a factor of 17.2x compared to naive MPC implementations.
  • Minimal Accuracy Loss: On MNIST, the validation accuracy only dropped from 99.04% (insecure) to 98.71% (secure/quantized) after 1,000 rounds.
  • Strong Defense: Against state-of-the-art "Min-Max" poisoning attacks, ScionFL-Aura limited the accuracy drop to just 7.7% for HSQ and 10.7% for KSQ, whereas such attacks can typically tank model accuracy by 20% or more.

Insights and Future Work

A key insight from this work is that mathematical precision in secure aggregation is not always necessary for machine learning. The inherent noise in ML training allows for cryptographic approximations like SepAgg to provide massive efficiency gains without compromising the final model. Furthermore, the authors discovered that 1-bit quantization actually enhances robustness by making it harder for attackers to craft subtle, malicious updates.

Future work includes extending the protocol to handle "malicious" servers (beyond the current semi-honest assumption) and providing formal theoretical proofs for the unbiased nature of SepAgg when combined with complex transforms like Kashin’s representation.

Summary updated

BibTeX
@inproceedings{benitzhak2024scionfl,
  author = {Yaniv Ben-Itzhak and Helen Möllering and Benny Pinkas and Thomas Schneider and Ajith Suresh and Oleksandr Tkachenko and Shay Vargaftik and Christian Weinert and Hossein Yalame and Avishay Yanai},
  title = {{ScionFL: Secure Quantized Aggregation for Federated Learning}},
  booktitle = {SaTML},
  year = {2024},
  url = {https://openreview.net/forum?id=0ypqmVIjD2},
}
Figure