Integrating Homomorphic Encryption and Trusted Execution Technology for Autonomous and Confidential Model Refining in Cloud

Pinglan Liu, Wensheng Zhang

TL;DR

This paper introduces a novel scheme for autonomous and confidential model refinement in the cloud by integrating Homomorphic Encryption (HE) and Trusted Execution Environments (TEEs). This approach allows cloud servers to continuously update machine learning models using new, encrypted data without exposing sensitive user data or model parameters, addressing critical privacy concerns in outsourced ML.

What Problem Does This Paper Address?

The increasing reliance on cloud computing for machine learning tasks, including model training and inference, presents a significant challenge: how to maintain the confidentiality of user data and proprietary models when outsourcing these processes. Users want to leverage the cloud's scalability and computational power, but they are concerned about potential data breaches or misuse by cloud providers. Furthermore, for long-term, continuous learning scenarios, users desire autonomous model refinement without constant manual intervention. Existing solutions like Multi-Party Computation (MPC), Homomorphic Encryption (HE), and Trusted Execution Environments (TEEs) each have limitations. MPC requires frequent interaction, HE is computationally expensive, and TEEs have limited resources and can be vulnerable to side-channel attacks. This paper aims to bridge this gap by combining the strengths of HE and TEEs to achieve secure, autonomous, and continuous model refinement.

Methods

The proposed scheme integrates two key technologies:

  • Homomorphic Encryption (HE): Specifically, a leveled homomorphic encryption (LHE) scheme (CKKS) is used. This allows computations (like model training operations) to be performed directly on encrypted data (ciphertexts) without decrypting them first. This ensures the confidentiality of both the model parameters and the training data. Ciphertext packing is also employed to improve efficiency by encoding multiple data points into a single ciphertext.
  • Trusted Execution Environment (TEE): A TEE (like Intel SGX) is utilized for specific, sensitive operations. Its primary roles are:
  • Key Management: Generating and distributing cryptographic keys for HE.
  • Initialization: Setting up the system securely.
  • Re-encryption: Periodically reducing the accumulated "noise" in HE ciphertexts. This noise is an inherent byproduct of HE computations and can limit the number of operations that can be performed. By re-encrypting, the noise is reset, allowing for continued computation. The TEE's involvement is minimized to reduce its attack surface and resource requirements.

The system architecture involves a cloud server with both a TEE and a Regular Execution Environment (REE), and clients (model providers and data providers). The process involves clients encrypting their models and data using keys provided by the TEE, sending them to the REE for processing, and the TEE handling sensitive operations like re-encryption. The paper details algorithms for both forward and backward propagation within the cloud server, adapted for encrypted computations.

Datasets

The experiments were conducted using the MNIST dataset for handwritten digit classification. The authors describe a specific experimental setup where a base model was initially trained on data dominated by odd-labeled images, and then refined using a new batch of data with a different distribution, where even-labeled images were dominant.

Key Insights and Ideas

  • Complementary Strengths: The core insight is that HE and TEEs are not mutually exclusive but rather complementary. HE provides robust data confidentiality during computation, while TEEs offer secure key management and a mechanism for noise reduction in HE, which is crucial for complex operations like model training.
  • Autonomous Refinement: The proposed system enables autonomous model refinement. Once set up, the cloud server can continuously update the model with new encrypted data without requiring the client to be online or actively participate in each refinement step.
  • Noise Management is Crucial: The paper highlights that noise accumulation is a more severe problem in model training/refining than in inference due to the backward propagation step. The TEE's re-encryption service is essential for managing this noise and making continuous training feasible.
  • Minimizing TEE Exposure: The design intentionally limits the TEE's functionality to key management and re-encryption, reducing its complexity and potential attack surface. Most of the heavy computation happens in the REE on encrypted data.

Key Results

  • Feasibility Demonstrated: The implementation and experiments confirm the feasibility of the proposed scheme. The cloud server can autonomously refine an encrypted model with new encrypted training data.
  • Accuracy Comparable to Baseline: The refined model achieved comparable accuracy to a baseline scheme that refines plaintext models with plaintext data, indicating that the confidentiality measures did not significantly degrade model performance over time.
  • Higher Computational Cost: As expected, the confidential scheme incurs significantly higher computational costs compared to the plaintext baseline. The paper notes that the current implementation uses a single CPU core and does not leverage GPUs, suggesting potential for substantial performance improvements.
  • Increased Communication Overhead: Encrypted data incurs higher communication overhead than plaintext data, though this communication primarily occurs at the initial setup phase.

Limitations and Future Work

  • Computational Efficiency: The most significant limitation acknowledged is the substantially lower computational efficiency compared to plaintext training. The authors attribute this to the overhead of HE operations and the lack of GPU utilization in their current setup.
  • Future Work: The authors plan to improve the scheme's performance by leveraging higher levels of parallelism and utilizing GPUs on the cloud server. They also suggest exploring further optimizations for the HE operations and TEE interactions.

Summary updated

Abstract

With the popularity of cloud computing and machine learning, it has been a trend to outsource machine learning processes (including model training and model-based inference) to cloud. By the outsourcing, other than utilizing the extensive and scalable resource offered by the cloud service provider, it will also be attractive to users if the cloud servers can manage the machine learning processes autonomously on behalf of the users. Such a feature will be especially salient when the machine learning is expected to be a long-term continuous process and the users are not always available to participate. Due to security and privacy concerns, it is also desired that the autonomous learning preserves the confidentiality of users' data and models involved. Hence, in this paper, we aim to design a scheme that enables autonomous and confidential model refining in cloud. Homomor-phic encryption and trusted execution environment technology can protect confidentiality for autonomous computation, but each of them has their limitations respectively and they are complementary to each other. Therefore, we further propose to integrate these two techniques in the design of the model refining scheme. Through implementation and experiments, we evaluate the feasibility of our proposed scheme. The results indicate that, with our proposed scheme the cloud server can autonomously re-fine an encrypted model with newly provided encrypted training data to continuously improve its accuracy. Though the efficiency is still significantly lower than the baseline scheme that refines plaintext-model with plaintext-data, we expect that it can be improved by fully utilizing the higher level of parallelism and the computational power of GPU at the cloud server.
BibTeX
@misc{liu2023integrating,
  author = {Pinglan Liu and Wensheng Zhang},
  title = {{Integrating Homomorphic Encryption and Trusted Execution Technology for Autonomous and Confidential Model Refining in Cloud}},
  year = {2023},
  eprint = {2308.00963},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.00963},
}
Figure