TL;DR
This paper introduces SHAPER, a novel hardware-software co-designed architecture that significantly accelerates privacy-preserving machine learning (PPML) by efficiently combining secret sharing (SS) and homomorphic encryption (HE). SHAPER achieves a remarkable 94x speedup over CPU clusters for large-scale logistic regression training, overcoming the performance bottlenecks that hinder the widespread adoption of PPML.
What Problem Does This Paper Address?
Machine learning (ML) models hold immense potential for analyzing sensitive data across various domains like healthcare and finance. However, privacy concerns and regulations often prevent direct use of this data, leading to the "isolated data island" problem. Privacy-preserving machine learning (PPML) aims to solve this by enabling collaborative model training and inference using techniques like multi-party computation (MPC) and homomorphic encryption (HE). While these methods protect raw data, they suffer from significant computational and network communication overheads, making them impractical for many real-world applications. Existing solutions often focus on either MPC or HE, or employ hybrid approaches with limited co-design of algorithms, protocols, and hardware. This paper addresses the critical need for a unified, efficient, and general-purpose architecture that can accelerate hybrid PPML protocols, specifically SS-AHE, by tackling these overheads through a holistic approach.
Methods
SHAPER employs a comprehensive algorithm-protocol-hardware co-design methodology. Key techniques include:
- Hybrid SS-AHE Architecture: SHAPER integrates Secret Sharing (SS) and Additive Homomorphic Encryption (AHE) to leverage their complementary strengths. SS handles data sparsity efficiently, while AHE provides encryption for secure computation.
- Hardware Acceleration: The architecture features dedicated hardware units for AHE (including optimized modular multiplication engines) and SS (including a high-throughput CSPRNG).
- Algorithm-Level Optimizations: Novel algorithms for modular multiplication (MM) and quick reduction (QR) are developed to improve the efficiency of core cryptographic operations.
- Protocol-Level Optimizations: Techniques like plaintext packing are used to reduce communication overhead by bundling multiple ciphertexts.
- Pipeline Execution: A pipelined execution process for MM operations is implemented to overlap computation and communication, hiding latency.
- Vectorized Operations: Support for vectorized operations allows for parallel processing of multiple data points, further boosting throughput.
- Instruction Set Architecture (ISA): A custom ISA is designed to efficiently map PPML operations to the hardware.
Datasets
Not discussed in the paper. The evaluation uses large-scale logistic regression training tasks, implying the use of synthetic or real-world datasets representative of such tasks, but specific datasets are not detailed.
Key Insights and Ideas
- Synergy of SS and HE: The paper highlights that combining SS and HE is crucial for achieving practical PPML. SS excels at handling sparse data, while HE provides robust privacy for computations.
- Holistic Co-design is Key: Significant performance gains are achieved by co-designing algorithms, protocols, and hardware, rather than optimizing each in isolation.
- Modular Multiplication is the Bottleneck: The efficiency of modular multiplication (MM) is identified as the primary performance bottleneck in AHE-based PPML. Optimizing MM is therefore paramount.
- Pipelining and Vectorization for Latency Hiding: Overlapping computation and communication through pipelining and parallelizing operations with vectorization are effective strategies for hiding latency.
- Hardware-Software Trade-offs: The design carefully balances hardware resource utilization (e.g., DSPs, URAMs) with performance gains, demonstrating a pragmatic approach to FPGA implementation.
Key Results
- 94x Speedup: SHAPER achieves a 94x speedup over CPU clusters for large-scale logistic regression training tasks compared to software-only benchmarks.
- Significant Performance Gains: The architecture demonstrates substantial improvements in function-level micro-benchmarks, with MM and ME efficiency improved by up to 46x compared to prior FPGA implementations.
- Efficient Cryptographic Primitives: Optimized MM and ME operations, along with CRT-based Paillier encryption/decryption, lead to significant acceleration.
- Scalability: SHAPER shows improved performance with larger sample sizes, indicating good scalability.
- FPGA Implementation: A prototype is successfully implemented on a Xilinx VU13P FPGA, showcasing the feasibility of the design.
Limitations and Future Work
- Focus on 2-Party Setting: SHAPER is primarily designed for 2-party PPML, which is common but not exhaustive. Future work could explore multi-party scenarios.
- Specific PPML Tasks: While SHAPER is designed to be general, the primary evaluation is on logistic regression. Its performance on other complex ML tasks like neural networks (NN) or XGBoost needs further exploration.
- Hardware Resource Utilization: While optimized, the design still consumes significant FPGA resources, particularly DSPs for MM engines. Future work might explore more resource-efficient MM designs.
- Security Parameter: The paper mentions a 2048/3072-bit Paillier system corresponding to 112/128-bit security. Further analysis of security implications for different parameter choices could be beneficial.
- Broader Application Scenarios: The authors suggest exploring SHAPER's applicability to other ML algorithms beyond logistic regression.
Summary updated