Abstract
Machine learning (ML) has been widely recognized as an enabler of the global trend of digital transformation. With the increasing adoption of data-hungry machine learning algorithms, personal data privacy has emerged as one of the key concerns that could hinder the success of digital transformation. As such, Privacy-Preserving Machine Learning (PPML) has received much attention from the machine learning community. Practical PPML typically allows multiple participants to individually train their ML models, which are then aggregated to construct a global model in a privacy-preserving manner. However, in large-scale applications like federated learning, participants are often resource-constrained mobile devices that may drop out due to connectivity or battery issues. Therefore, the resilience of privacy-preserving aggregation has become a critical problem. In this paper, we propose a scalable privacy-preserving aggregation scheme that can tolerate dropout by participants at any time and is secure against both semi-honest and active malicious adversaries. By replacing communication-intensive building blocks with a seed homomorphic pseudo-random generator and relying on the additive homomorphic property of Shamir secret sharing, our scheme outperforms state-of-the-art schemes by up to $6.37\times$ in runtime and provides stronger dropout resilience.
The Challenge: Privacy vs. Reliability in Federated Learning
In Federated Learning (FL), a central server aggregates model updates (gradients) from thousands of distributed clients to train a global model without ever seeing raw data. To ensure privacy, "Secure Aggregation" protocols mask these updates so the server only sees the final sum. However, real-world FL faces two major hurdles:
- Dropout: Mobile devices frequently go offline. Existing protocols like Google’s SecAgg handle this by having the server reconstruct the masks of dropped-out users, a process that becomes computationally expensive as the dropout rate increases.
- Overhead: Standard protocols often rely on pairwise Diffie-Hellman key exchanges, creating $O(n^2)$ communication complexity, which makes scaling to very large networks difficult.
A Streamlined Approach to Secure Aggregation
The proposed scheme shifts away from pairwise masking toward a more streamlined "seed-based" approach. Key components include a Homomorphic Pseudorandom Generator (HPRG), which allows the server to reconstruct a single "aggregate mask" for all users at once, and Shamir Secret Sharing, where clients share pieces of their individual seeds. If a client drops out, the remaining clients provide enough shares for the server to reconstruct the necessary parts of the aggregate mask.
As shown in Table 1, the proposed protocol achieves a server computation complexity of $O(n)$, a significant improvement over the $O(mn^2)$ or $O(mn \log n)$ complexities found in earlier works like SecAgg and TurboAgg. Furthermore, it maintains resilience against malicious adversaries while requiring only three communication rounds.
Key Insights: The "Inverse Dropout" Advantage
A unique property of this scheme is its efficiency regarding dropouts. In almost all previous secure aggregation schemes, more dropouts mean more work for the server to reconstruct individual masks. In this protocol, the server only performs one reconstruction for the final mask. Consequently, as more clients drop out, there are fewer shares to process, and the total runtime actually decreases.
Figure 1 illustrates the performance bottleneck of existing methods: as the dropout rate increases from 0% to 30%, the total run time for protocols like SecAgg and TurboAgg increases significantly. By contrast, the proposed method eliminates these pairwise bottlenecks, making the protocol much more attractive for resource-constrained mobile environments.
Experimental Results
Evaluation using synthetic gradient vectors matching models like MobileNet V3 and ResNet18 yielded the following results:
- Speed: The scheme is up to 6.37× faster than SecAgg and its variants in total runtime.
- Training Time: For a CIFAR-10 classification task, the scheme completed training in 1.67 hours, compared to 175 hours required by the POSEIDON framework under similar conditions.
- Scalability: In a WAN setting with 1,000 clients and a 30% dropout rate, the total runtime was approximately 101 seconds.
- Communication: Each client only needs to send $O(m + n)$ data, keeping mobile-side overhead low.
Limitations and Future Work
While highly efficient, the protocol faces a bottleneck when solving discrete logarithms for massive models; for ResNet18 (11.7M parameters), this step takes over 2,000 seconds. Additionally, while the protocol protects gradients during aggregation, the final global model remains visible, leaving it potentially vulnerable to membership inference attacks. Future work will focus on integrating defense metrics against model poisoning and optimizing discrete logarithm recovery for billion-parameter models.
Summary updated