Meteor: Improved Secure 3-Party Neural Network Inference with Reducing Online Communication Costs

Ye Dong, Xiaojun Chen, Weizhan Jing, Kaiyun Li, Weiping Wang

TL;DR

This paper introduces METEOR, a novel 3-party secure computation protocol for neural network inference that significantly reduces online communication costs. By proposing an improved secret sharing scheme and optimizing core cryptographic primitives, METEOR achieves substantial speedups and communication savings compared to existing state-of-the-art methods, making private AI inference more practical.

What Problem Does This Paper Address?

The increasing adoption of Deep-Learning-as-a-Service (DLaaS) raises significant privacy concerns, as users must send their sensitive data to a service provider for inference. Secure inference protocols aim to address this by allowing computations on encrypted or shared data, ensuring only the final result is revealed. However, existing secure inference methods, particularly those based on secret sharing, suffer from high communication overhead during the online phase (when actual inference occurs). This overhead, especially in Wide Area Network (WAN) settings, limits the practical applicability and throughput of these systems. METEOR tackles this challenge by focusing on optimizing the online communication costs of secure neural network inference.

Methods

METEOR's core innovation lies in its improved 3-party secret sharing scheme, termed "[-]-sharing." This scheme leverages the linearity of replicated secret sharing to design more efficient protocols for fundamental cryptographic operations. Key methodological contributions include:

  • Improved 3-Party Secret Sharing ([·]-sharing): Unlike traditional 3-out-of-3 or 2-out-of-3 secret sharing, METEOR's [-]-sharing allows for more efficient handling of intermediate results, particularly for multiplication.
  • Efficient Cryptographic Primitives: The paper details optimized protocols for essential operations like linear operations (Lops), multiplication (2-MULT and N-MULT), most significant bit (MSB) extraction, and multiplexers (MUX). A key advancement is the "free truncation" technique for fixed-point multiplication, which significantly reduces communication overhead.
  • Optimized Neural Network Operators: METEOR builds secure blocks for common NN layers such as Matrix Multiplication (MatMul), ReLU, and Max Pooling (MP), integrating the efficient primitives. These blocks are designed with specific optimizations to further enhance online efficiency.
  • Function-Dependent, Input-Independent Setup: Similar to prior work, METEOR separates computation into a setup phase (generating correlated randomness) and an online phase (performing inference). This allows for efficient online execution by pre-computing necessary resources.

Datasets

The paper evaluates METEOR on standard machine learning datasets:

  • MNIST: A widely used dataset for digit recognition.
  • CIFAR-10: A dataset of small color images in 10 classes.

The authors also use neural network architectures from both the secure ML community (Network-A, B, C) and the general ML community (LeNet, AlexNet, VGG16) to demonstrate the effectiveness of their approach.

Key Insights and Ideas

  • Linearity is Key for Efficiency: The paper demonstrates that by carefully exploiting the linearity property of secret sharing, significant gains in online communication can be achieved, especially for multiplication operations.
  • "Free Truncation" Revolutionizes Fixed-Point Arithmetic: The novel "free truncation" technique for fixed-point multiplication is a major breakthrough, allowing for accurate inference without the high communication costs previously associated with this operation.
  • Optimizing Non-Linear Functions is Crucial: While linear operations are relatively efficient, non-linear functions like ReLU and Max Pooling often become bottlenecks. METEOR's efficient protocols for MSB extraction and multiplexing directly address these bottlenecks.
  • 3PC Offers a Sweet Spot for Efficiency: The paper reinforces that 3-party computation, with an honest majority, provides a good balance between security and efficiency, outperforming 2PC in certain aspects while being more practical than higher-party settings for this specific problem.

Key Results

METEOR demonstrates significant improvements over existing methods like SecureNN and FALCON:

  • Online Communication Reduction:
    • Up to 25.6x reduction for single inference on MNIST compared to SecureNN.
  • Up to 1.5x reduction compared to FALCON.
  • Approximately 1.2-6x improvement for linear operators (MatMul, CONV) and 1.6-2.5x for non-linear operators (ReLU, MP).
  • Running-Time Improvements:
    • Up to 9.8x speedup for single inference on MNIST in LAN compared to SecureNN.
  • Up to 8.1x speedup in WAN compared to SecureNN.
  • Up to 1.5x speedup in LAN and 2.1x in WAN compared to FALCON.
  • For batch inference, METEOR shows better scalability, achieving comparable efficiency in LAN and ~1.5x improvements in WAN over FALCON.

Limitations and Future Work

  • Setup Phase Costs: While METEOR excels in online efficiency, its setup phase communication costs are noted to be slightly larger than SecureNN and FALCON. However, the authors argue this is a worthwhile trade-off for optimizing the critical online phase.
  • Semi-Honest Adversary Model: The current security model assumes a semi-honest adversary. Extending to a malicious adversary model would require additional cryptographic overhead.
  • Future Work: The authors suggest improving other MSB extraction methods and further optimizing setup communication costs. They also mention the potential of their [-]-sharing scheme for other applications like privacy-preserving e-voting.

Summary updated

Abstract

Secure neural network inference has been a promising solution to private Deep-Learning-as-a-Service, which enables the service provider and user to execute neural network inference without revealing their private inputs. However, the expensive overhead of current schemes is still an obstacle when applied in real applications. In this work, we present Meteor, an online communication-efficient and fast secure 3-party computation neural network inference system aginst semi-honest adversary in honest-majority. The main contributions of Meteor are two-fold: i) We propose a new and improved 3-party secret sharing scheme stemming from the linearity of replicated secret sharing, and design efficient protocols for the basic cryptographic primitives, including linear operations, multiplication, most significant bit extraction, and multiplexer. ii) Furthermore, we build efficient and secure blocks for the widely used neural network operators such as Matrix Multiplication, ReLU, and Maxpool, along with exploiting several specific optimizations for better efficiency. Our total communication with the setup phase is a little larger than SecureNN (PoPETs’19) and Falcon (PoPETs’21), two state-of-the-art solutions, but the gap is not significant when the online phase must be optimized as a priority. Using Meteor, we perform extensive evaluations on various neural networks. Compared to SecureNN and Falcon, we reduce the online communication costs by up to 25.6 × and 1.5 ×, and improve the running-time by at most 9.8 × (resp. 8.1 ×) and 1.5 × (resp. 2.1 ×) in LAN (resp. WAN) for the online inference.
BibTeX
@article{dong2023meteor,
  author = {Ye Dong and Xiaojun Chen and Weizhan Jing and Kaiyun Li and Weiping Wang},
  title = {{Meteor: Improved Secure 3-Party Neural Network Inference with Reducing Online Communication Costs}},
  journal = {IACR Cryptol. ePrint Arch.},
  year = {2023},
  doi = {10.1145/3543507.3583272},
}
Figure