TL;DR
This paper introduces SHARK, a novel system for actively secure two-party machine learning inference. By extending Function Secret Sharing (FSS) with new techniques, SHARK achieves significant speedups (up to 2300x) and reduced communication compared to prior state-of-the-art methods, making secure ML inference more practical.
What Problem Does This Paper Address?
Securely performing machine learning inference, where two parties (say, a model owner and a data holder) want to compute a model's output on private data without revealing the data or the model, is a critical area of research. While many protocols exist, most focus on "semi-honest" security, assuming parties follow the protocol but might try to learn extra information. However, real-world scenarios often require "active security," where parties might deviate maliciously from the protocol. Existing actively secure protocols for this task, particularly those using function secret sharing (FSS), suffer from significant overheads in terms of communication, rounds of interaction, and preprocessing material, making them impractical. This paper aims to bridge this gap by developing an actively secure FSS-based inference protocol that is significantly more efficient.
Methods
The core of SHARK's efficiency comes from several key innovations:
- Interactive FSS (IFSS): The authors introduce a generalization of FSS that allows for limited interaction between parties during computation. This is crucial for handling complex functions more efficiently than traditional, non-interactive FSS.
- Hybrid Boolean and Arithmetic Values: Unlike prior work that strictly separated boolean and arithmetic computations, SHARK allows for a mix. This is achieved by leveraging authenticated boolean secret sharing schemes, enabling more streamlined processing of functions that inherently involve both types of values (e.g., comparisons).
- Optimized FSS Sub-protocols: The paper presents new, efficient FSS-based sub-protocols for common ML operations like ReLU, arithmetic and logical right shifts (ARS/LRS), a fused ReluARS operation, and splines (used for approximating complex functions). These are designed to work with the new IFSS framework and the hybrid value approach.
- Leveraging Preprocessing: Like many secure computation protocols, SHARK relies on a preprocessing phase where parties obtain correlated randomness. The key is that SHARK uses this material much more efficiently.
- "Interactive FSS" for Malicious Security: The authors adapt FSS for active security by using a technique that allows parties to hold masked, authenticated values. This is a departure from previous approaches that either required larger masked values or were not applicable to all FSS-based protocols.
Datasets
The paper evaluates SHARK on standard machine learning benchmarks, including:
- Feed-forward Neural Networks (FFNN): D4 (Benchmark 4 of DeepSecure) for smart-sensing applications.
- Convolutional Neural Networks (CNNs): M1 (2-layer CNN for MNIST), M2 (7-layer CNN for CIFAR-10), HiNet (3-layer CNN for CIFAR-10), AlexNet (8-layer CNN for ImageNet), and VGG16 (16-layer CNN for ImageNet).
- Transformers: BERT (specifically, the BERT-Base model from MPCFormer) for natural language processing tasks.
No new benchmarks were introduced; instead, the authors used existing datasets and models to compare against prior work.
Key Insights and Ideas
- Interactive FSS is Key: The introduction of interactive FSS is a significant conceptual leap, allowing for more flexible and efficient computation of complex functions in the malicious setting.
- Mixing Boolean and Arithmetic is More Efficient: By cleverly combining boolean and arithmetic secret sharing, SHARK avoids the overheads of strictly separating these computations, leading to better performance.
- Active Security Doesn't Have to Be That Slow: The paper demonstrates that achieving active security for ML inference doesn't necessarily require the massive overheads previously seen, thanks to novel FSS adaptations.
- Specialized Sub-protocols Matter: Developing optimized FSS schemes for specific ML primitives like ReLU, ARS, and splines is crucial for overall system efficiency.
- Transformers are Now Actively Secure: SHARK enables the first actively secure inference for transformer models, a significant advancement for privacy-preserving NLP.
Key Results
- Speedup: SHARK outperforms the state-of-the-art actively secure protocol (Escudero et al. [29]) by up to 2300x in runtime and 320x in communication on various ML benchmarks.
- Reduced Communication and Rounds: Compared to prior work, SHARK uses significantly less communication and fewer rounds of interaction.
- Lower Preprocessing Material: Surprisingly, SHARK also uses less preprocessing material than previous actively secure FSS-based protocols.
- Transformer Inference: SHARK enables the first actively secure inference for transformer models, with only a modest overhead (less than 3x) compared to state-of-the-art semi-honest solutions.
- Microbenchmarks: Specific functions like ReLU, ReluARS, and splines show substantial improvements, with speedups ranging from 4x to over 200x depending on the function and network setting (LAN vs. WAN).
Limitations and Future Work
- Theoretical vs. Practical: While the paper presents a practical implementation (SHARK), the theoretical framework for adapting FSS to active security [6] had limitations regarding arbitrary FSS protocols and required larger input sizes. SHARK addresses these limitations.
- Approximation for Reciprocal: For the reciprocal function, SHARK uses an approximation based on other FSS schemes rather than a direct FSS protocol. This is a practical trade-off for efficiency.
- Future Work: The paper implicitly suggests that further optimizations and extensions of IFSS could lead to even more efficient secure inference protocols. The focus on transformers also opens doors for more advanced privacy-preserving NLP applications.
Summary updated