BAYHENN: Combining Bayesian Deep Learning and Homomorphic Encryption for Secure DNN Inference

Peichen Xie, Bingzhe Wu, Guangyu Sun

Abstract

Recently, deep learning as a service (DLaaS) has emerged as a promising way to facilitate the employment of deep neural networks (DNNs) for various purposes. However, using DLaaS also causes potential privacy leakage from both clients and cloud servers. This privacy issue has fueled the research interests on the privacy-preserving inference of DNN models in the cloud service. In this paper, we present a practical solution named BAYHENN for secure DNN inference. It can protect both the client's privacy and server's privacy at the same time. The key strategy of our solution is to combine homomorphic encryption and Bayesian neural networks. Specifically, we use homomorphic encryption to protect a client's raw data and use Bayesian neural networks to protect the DNN weights in a cloud server. To verify the effectiveness of our solution, we conduct experiments on MNIST and a real-life clinical dataset. Our solution achieves consistent latency decreases on both tasks. In particular, our method can outperform the best existing method (GAZELLE) by about 5x, in terms of end-to-end latency.

Introduction

The rise of Deep Learning as a Service (DLaaS) has created a significant privacy challenge. On one hand, clients must upload sensitive data (e.g., medical records or personal images) to the cloud. On the other hand, service providers must protect their proprietary model weights, which represent significant intellectual property and are vulnerable to reverse-engineering.

Existing solutions often rely on Homomorphic Encryption (HE) or Multi-Party Computation (MPC). While HE allows computation on encrypted data, it is computationally expensive and struggles with non-linear activation functions like ReLU. Interactive protocols attempt to solve this by splitting tasks, but they often leak intermediate information that allows a malicious client to reconstruct the server's model.

The BAYHENN Framework

BAYHENN addresses these limitations by combining a vectorizable HE scheme with Bayesian Neural Networks (BNNs). The protocol is divided into two main components:

  1. Secure Linear Computation (SLC): The client encrypts their input using the BFV homomorphic encryption scheme. The server then performs linear operations (convolutions and fully connected layers) directly on the ciphertext.
  2. Secure Non-linear Computation (SNC): To handle non-linearities without the overhead of Garbled Circuits, BAYHENN allows the client to decrypt intermediate results, apply the activation function locally, and re-encrypt the data for the next layer.
Comparison of privacy features and activation function support across different frameworks.

Protecting Server Privacy with BNNs

The primary risk of the SNC phase is that the client sees intermediate feature maps. To mitigate this, BAYHENN utilizes Bayesian Neural Networks. Instead of static weights, BNN weights are represented as probability distributions. During inference, the server samples weights, effectively injecting "proper randomness" into the computation. This transforms the model-stealing attempt into a Learning With Errors (LWE) problem, which is computationally infeasible to solve.

Experimental Evaluation

The authors evaluated BAYHENN on the MNIST handwritten digit dataset and the Invasive Ductal Carcinoma (IDC) clinical dataset.

Performance and Latency

BAYHENN demonstrates a significant performance leap over existing frameworks like GAZELLE and CryptoNets. On the MNIST task, BAYHENN achieved an end-to-end latency of 1.36 seconds, a nearly 5x improvement over GAZELLE's 6.29 seconds.

Performance comparison on the MNIST dataset, highlighting BAYHENN's latency advantage.

Communication Efficiency

By avoiding heavy cryptographic primitives for non-linear layers, BAYHENN drastically reduces communication overhead. For an AlexNet model on the IDC dataset, communication dropped from 252.6 MB (GAZELLE) to just 6.32 MB. This efficiency is critical for real-world deployment where bandwidth may be limited.

Accuracy and Generalization

The accuracy trade-off is minimal. On MNIST, BAYHENN achieved 98.93% accuracy compared to the 99.05% of non-private models. Furthermore, the Bayesian approach acted as a regularizer, reducing the gap between training and testing accuracy, which suggests better generalization and less overfitting compared to standard deterministic models.

Conclusion

BAYHENN provides a practical, high-performance solution for secure DLaaS. By integrating BNNs with HE, it achieves a "two-sided" privacy guarantee while outperforming state-of-the-art methods in both latency and communication costs. Future work will focus on optimizing BNN training and exploring hardware acceleration to further reduce encryption overhead.

Summary updated

Abstract

Recently, deep learning as a service (DLaaS) has emerged as a promising way to facilitate the employment of deep neural networks (DNNs) for various purposes. However, using DLaaS also causes potential privacy leakage from both clients and cloud servers. This privacy issue has fueled the research interests on the privacy-preserving inference of DNN models in the cloud service. In this paper, we present a practical solution named BAYHENN for secure DNN inference. It can protect both the client's privacy and server's privacy at the same time. The key strategy of our solution is to combine homomorphic encryption and Bayesian neural networks. Specifically, we use homomorphic encryption to protect a client's raw data and use Bayesian neural networks to protect the DNN weights in a cloud server. To verify the effectiveness of our solution, we conduct experiments on MNIST and a real-life clinical dataset. Our solution achieves consistent latency decreases on both tasks. In particular, our method can outperform the best existing method (GAZELLE) by about 5x, in terms of end-to-end latency.
BibTeX
@inproceedings{xie2019bayhenn,
  author = {Peichen Xie and Bingzhe Wu and Guangyu Sun},
  title = {{BAYHENN: Combining Bayesian Deep Learning and Homomorphic Encryption for Secure DNN Inference}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=pvQtfM4tfb},
}
Figure