STAMP: Lightweight TEE-Assisted MPC for Efficient Privacy-Preserving Machine Learning

Pengzhi Huang, Thang Hoang, Yueying Li, Elaine Shi, G. Edward SuharXiv 2022 · DOI · OpenReview

TL;DR

This paper introduces STAMP, a novel framework for privacy-preserving machine learning (PPML) that leverages small, dedicated "lightweight trusted hardware" (LTH) – like a TPM or an Apple Secure Enclave – to accelerate secure multi-party computation (MPC) protocols. STAMP achieves significant speedups (4x-63x) over existing MPC methods by offloading computationally intensive non-linear operations to the LTH, making secure ML inference more efficient and practical.

What Problem Does This Paper Address?

Machine learning (ML) models are increasingly used for tasks involving sensitive data, raising privacy concerns. While techniques like secure multi-party computation (MPC) and trusted execution environments (TEEs) offer solutions, they often come with substantial performance overheads. Traditional TEEs, integrated into high-performance processors, are complex, have large attack surfaces, and can be vulnerable. Existing MPC protocols, while offering strong security, are computationally expensive, especially for complex neural networks. The challenge is to find a way to achieve efficient and secure ML inference without the drawbacks of current approaches.

Methods

STAMP's core innovation lies in its hybrid approach, combining MPC with lightweight trusted hardware (LTH). Instead of relying on high-performance TEEs for computation, STAMP uses LTHs (comparable to TPMs or on-chip security modules) to accelerate specific, computationally intensive non-linear operations (like ReLU, Softmax) within MPC protocols. The bulk of the computation, particularly linear operations, is still handled by standard, untrusted CPUs/GPUs via MPC. This strategy is effective because non-linear operations, while simple in plaintext, become major bottlenecks in MPC. STAMP carefully redesigns MPC protocols to efficiently offload these non-linear parts to the LTH, minimizing communication overhead between the LTH and the untrusted host. The paper also details protocols for specific operations like ReLU, matrix multiplication combined with ReLU, and Softmax, and addresses security guarantees against malicious adversaries under an honest majority assumption.

Datasets

The paper evaluates STAMP on several standard datasets:

  • MNIST
  • CIFAR-10
  • ImageNet
  • Wikitext-2

Key Insights and Ideas

  • Small Trusted Hardware is Sufficient: The most striking insight is that even very limited, low-performance trusted hardware (like a TPM) can significantly boost the performance of privacy-preserving ML if the MPC protocol is carefully designed to leverage it for specific tasks.
  • Non-linear Operations are the Bottleneck: The paper highlights that in MPC-based ML inference, non-linear operations (e.g., ReLU, Softmax) are the primary performance bottleneck, not linear operations. This is counter-intuitive compared to plaintext ML.
  • Hybrid Approach Wins: Combining the strengths of MPC (strong security guarantees) with the efficiency of LTH for specific operations offers a compelling trade-off between security and performance.
  • LTH Benefits Beyond Performance: LTHs offer advantages in terms of a smaller trusted computing base (TCB), reduced complexity, and potentially better physical isolation compared to high-performance TEEs.

Key Results

  • Significant Speedups: STAMP achieves substantial performance improvements over state-of-the-art distributed PPML protocols. In the semi-honest WAN/GPU setting, it's 4x-63x faster than Falcon and AriaNN.
  • Communication Efficiency: STAMP also reduces inter-party communication by 3.8x-12x.
  • Malicious Security: The platform provides security guarantees against malicious adversaries under an honest majority assumption.
  • Scalability: STAMP is not limited by the secure memory size of a TEE and can support large neural networks like ResNet18 and Transformers.
  • Comparison to TEEs: Interestingly, STAMP with a powerful LTH-SoC can even outperform protocols using high-performance TEEs (like Intel SGX) with GPU outsourcing, demonstrating the effectiveness of offloading non-linear operations.

Limitations and Future Work

  • LTH Performance: While STAMP significantly improves performance, the overall speedup is still dependent on the LTH's capabilities. For very large models or specific network architectures, the LTH can still become a bottleneck, especially with low-bandwidth interconnections (e.g., LTH-chip).
  • Model Architecture Dependence: The speedup achieved by STAMP varies with the ML model's architecture, being more pronounced for models with frequent and complex non-linear operations (like Transformers).
  • Future Work: The authors suggest integrating STAMP into existing SoCs to better understand LTH capabilities, extending STAMP to handle larger models like LLMs and diffusion models, and further optimizing protocols to reduce LTH overhead. They also propose exploring the use of LTH for other types of MPC protocols and operations.

Summary updated

Abstract

In this paper, we propose a new secure machine learning inference platform assisted by a small dedicated security processor, which will be easier to protect and deploy compared to today's TEEs integrated into high-performance processors. Our platform provides three main advantages over the state-of-the-art: (i) We achieve significant performance improvements compared to state-of-the-art distributed Privacy-Preserving Machine Learning (PPML) protocols, with only a small security processor that is comparable to a discrete security chip such as the Trusted Platform Module (TPM) or on-chip security subsystems in SoCs similar to the Apple enclave processor. In the semi-honest setting with WAN/GPU, our scheme is 4X-63X faster than Falcon (PoPETs'21) and AriaNN (PoPETs'22) and 3.8X-12X more communication efficient. We achieve even higher performance improvements in the malicious setting. (ii) Our platform guarantees security with abort against malicious adversaries under honest majority assumption. (iii) Our technique is not limited by the size of secure memory in a TEE and can support high-capacity modern neural networks like ResNet18 and Transformer. While previous work investigated the use of high-performance TEEs in PPML, this work represents the first to show that even tiny secure hardware with really limited performance can be leveraged to significantly speed-up distributed PPML protocols if the protocol can be carefully designed for lightweight trusted hardware.
BibTeX
@misc{huang2022stamp,
  author = {Pengzhi Huang and Thang Hoang and Yueying Li and Elaine Shi and G. Edward Suh},
  title = {{STAMP: Lightweight TEE-Assisted MPC for Efficient Privacy-Preserving Machine Learning}},
  year = {2022},
  eprint = {2210.10133},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.10133},
}
Figure