TL;DR
This paper introduces METEOR, a novel 3-party secure computation protocol for neural network inference that significantly reduces online communication costs. By proposing an improved secret sharing scheme and optimizing core cryptographic primitives, METEOR achieves substantial speedups and communication savings compared to existing state-of-the-art methods, making private AI inference more practical.
What Problem Does This Paper Address?
The increasing adoption of Deep-Learning-as-a-Service (DLaaS) raises significant privacy concerns, as users must send their sensitive data to a service provider for inference. Secure inference protocols aim to address this by allowing computations on encrypted or shared data, ensuring only the final result is revealed. However, existing secure inference methods, particularly those based on secret sharing, suffer from high communication overhead during the online phase (when actual inference occurs). This overhead, especially in Wide Area Network (WAN) settings, limits the practical applicability and throughput of these systems. METEOR tackles this challenge by focusing on optimizing the online communication costs of secure neural network inference.
Methods
METEOR's core innovation lies in its improved 3-party secret sharing scheme, termed "[-]-sharing." This scheme leverages the linearity of replicated secret sharing to design more efficient protocols for fundamental cryptographic operations. Key methodological contributions include:
- Improved 3-Party Secret Sharing ([·]-sharing): Unlike traditional 3-out-of-3 or 2-out-of-3 secret sharing, METEOR's [-]-sharing allows for more efficient handling of intermediate results, particularly for multiplication.
- Efficient Cryptographic Primitives: The paper details optimized protocols for essential operations like linear operations (Lops), multiplication (2-MULT and N-MULT), most significant bit (MSB) extraction, and multiplexers (MUX). A key advancement is the "free truncation" technique for fixed-point multiplication, which significantly reduces communication overhead.
- Optimized Neural Network Operators: METEOR builds secure blocks for common NN layers such as Matrix Multiplication (MatMul), ReLU, and Max Pooling (MP), integrating the efficient primitives. These blocks are designed with specific optimizations to further enhance online efficiency.
- Function-Dependent, Input-Independent Setup: Similar to prior work, METEOR separates computation into a setup phase (generating correlated randomness) and an online phase (performing inference). This allows for efficient online execution by pre-computing necessary resources.
Datasets
The paper evaluates METEOR on standard machine learning datasets:
- MNIST: A widely used dataset for digit recognition.
- CIFAR-10: A dataset of small color images in 10 classes.
The authors also use neural network architectures from both the secure ML community (Network-A, B, C) and the general ML community (LeNet, AlexNet, VGG16) to demonstrate the effectiveness of their approach.
Key Insights and Ideas
- Linearity is Key for Efficiency: The paper demonstrates that by carefully exploiting the linearity property of secret sharing, significant gains in online communication can be achieved, especially for multiplication operations.
- "Free Truncation" Revolutionizes Fixed-Point Arithmetic: The novel "free truncation" technique for fixed-point multiplication is a major breakthrough, allowing for accurate inference without the high communication costs previously associated with this operation.
- Optimizing Non-Linear Functions is Crucial: While linear operations are relatively efficient, non-linear functions like ReLU and Max Pooling often become bottlenecks. METEOR's efficient protocols for MSB extraction and multiplexing directly address these bottlenecks.
- 3PC Offers a Sweet Spot for Efficiency: The paper reinforces that 3-party computation, with an honest majority, provides a good balance between security and efficiency, outperforming 2PC in certain aspects while being more practical than higher-party settings for this specific problem.
Key Results
METEOR demonstrates significant improvements over existing methods like SecureNN and FALCON:
- Online Communication Reduction:
- Up to 25.6x reduction for single inference on MNIST compared to SecureNN.
- Up to 1.5x reduction compared to FALCON.
- Approximately 1.2-6x improvement for linear operators (MatMul, CONV) and 1.6-2.5x for non-linear operators (ReLU, MP).
- Running-Time Improvements:
- Up to 9.8x speedup for single inference on MNIST in LAN compared to SecureNN.
- Up to 8.1x speedup in WAN compared to SecureNN.
- Up to 1.5x speedup in LAN and 2.1x in WAN compared to FALCON.
- For batch inference, METEOR shows better scalability, achieving comparable efficiency in LAN and ~1.5x improvements in WAN over FALCON.
Limitations and Future Work
- Setup Phase Costs: While METEOR excels in online efficiency, its setup phase communication costs are noted to be slightly larger than SecureNN and FALCON. However, the authors argue this is a worthwhile trade-off for optimizing the critical online phase.
- Semi-Honest Adversary Model: The current security model assumes a semi-honest adversary. Extending to a malicious adversary model would require additional cryptographic overhead.
- Future Work: The authors suggest improving other MSB extraction methods and further optimizing setup communication costs. They also mention the potential of their [-]-sharing scheme for other applications like privacy-preserving e-voting.
Summary updated