BAYHENN: Combining Bayesian Deep Learning and Homomorphic Encryption for Secure DNN Inference

Peichen Xie, Bingzhe Wu, Guangyu Sun

Abstract

Recently, deep learning as a service (DLaaS) has emerged as a promising way to facilitate the employment of deep neural networks (DNNs) for various purposes. However, using DLaaS also causes potential privacy leakage from both clients and cloud servers. This privacy issue has fueled the research interests on the privacy-preserving inference of DNN models in the cloud service. In this paper, we present a practical solution named BAYHENN for secure DNN inference. It can protect both the client's privacy and server's privacy at the same time. The key strategy of our solution is to combine homomorphic encryption and Bayesian neural networks. Specifically, we use homomorphic encryption to protect a client's raw data and use Bayesian neural networks to protect the DNN weights in a cloud server. To verify the effectiveness of our solution, we conduct experiments on MNIST and a real-life clinical dataset. Our solution achieves consistent latency decreases on both tasks. In particular, our method can outperform the best existing method (GAZELLE) by about 5x, in terms of end-to-end latency.

Introduction: The Privacy Dilemma in DLaaS

As Deep Learning as a Service (DLaaS) becomes ubiquitous, a significant privacy conflict has emerged. Clients wish to utilize powerful cloud-based models without exposing sensitive raw data, such as medical records or private photos. Conversely, service providers must protect their proprietary model weights, which represent valuable intellectual property.

Existing solutions often struggle to balance security, speed, and functionality. Pure Homomorphic Encryption (HE) allows computation on encrypted data but is computationally prohibitive and fails to support non-linear activation functions like ReLU. Interactive paradigms improve performance by delegating non-linear computations to the client, but this often leaks intermediate features that a malicious client could use to reverse-engineer and "steal" the server's model.

Comparison of privacy features and activation function support across frameworks.

Methodology: The BAYHENN Framework

The proposed BAYHENN framework addresses these challenges by combining HE with Bayesian Neural Networks (BNNs). The protocol is divided into two primary components:

  1. Secure Linear Computation (SLC): The client encrypts data using the BFV homomorphic encryption scheme. The server then performs linear operations, such as convolutions, directly on the ciphertext.
  2. Secure Non-linear Computation (SNC): The server returns the obscured results to the client. The client decrypts, applies the activation function (e.g., ReLU or Sigmoid), and re-encrypts the data for subsequent layers.

Bayesian Weights as a Cryptographic Shield

To prevent model theft during the SNC phase, BAYHENN replaces fixed weights with BNNs. Instead of deterministic values, weights are represented as probability distributions (mean and variance). During inference, the server samples multiple weight sets, introducing controlled "noise" into the intermediate features. The authors demonstrate that extracting the original weights from this noisy data is as difficult as solving the Learning With Errors (LWE) problem—a foundation of lattice-based cryptography considered NP-hard.

Evaluation and Results

The framework was evaluated using the MNIST handwritten digit dataset and the Invasive Ductal Carcinoma (IDC) clinical dataset for breast cancer classification.

Performance and Efficiency

BAYHENN demonstrates superior performance compared to state-of-the-art frameworks like GAZELLE and CryptoNets. On the MNIST benchmark, BAYHENN achieved an end-to-end latency of 1.36 seconds, a roughly 4.6x speedup over GAZELLE (6.29s) and a massive improvement over CryptoNets (3593s).

Communication overhead (MB) comparison between frameworks on MNIST and IDC datasets.

Furthermore, BAYHENN is significantly more communication-efficient. On the IDC dataset using AlexNet, it required only 6.32 MB of communication, compared to the 252.60 MB required by GAZELLE. This efficiency stems from avoiding heavy cryptographic primitives like Garbled Circuits for non-linear layers, which create massive communication overhead.

Accuracy and Robustness

The privacy enhancements do not significantly compromise accuracy. On MNIST, accuracy only dropped from 99.05% (standard) to 98.93%. Interestingly, the stochastic nature of BNNs provides a natural regularization effect, resulting in a smaller "generalization gap" (the difference between training and testing performance) than standard models, making it more robust against overfitting.

Conclusion

BAYHENN offers a practical path forward for secure DLaaS by repurposing Bayesian uncertainty as a security feature. By achieving significant speedups and communication reductions while maintaining high accuracy, it makes real-time secure inference more feasible. Future work includes optimizing training with specialized optimizers, scaling to larger architectures like ResNets, and exploring hardware acceleration to further minimize latency.

Summary updated

Abstract

Recently, deep learning as a service (DLaaS) has emerged as a promising way to facilitate the employment of deep neural networks (DNNs) for various purposes. However, using DLaaS also causes potential privacy leakage from both clients and cloud servers. This privacy issue has fueled the research interests on the privacy-preserving inference of DNN models in the cloud service. In this paper, we present a practical solution named BAYHENN for secure DNN inference. It can protect both the client's privacy and server's privacy at the same time. The key strategy of our solution is to combine homomorphic encryption and Bayesian neural networks. Specifically, we use homomorphic encryption to protect a client's raw data and use Bayesian neural networks to protect the DNN weights in a cloud server. To verify the effectiveness of our solution, we conduct experiments on MNIST and a real-life clinical dataset. Our solution achieves consistent latency decreases on both tasks. In particular, our method can outperform the best existing method (GAZELLE) by about 5x, in terms of end-to-end latency.
BibTeX
@inproceedings{xie2019bayhenn,
  author = {Peichen Xie and Bingzhe Wu and Guangyu Sun},
  title = {{BAYHENN: Combining Bayesian Deep Learning and Homomorphic Encryption for Secure DNN Inference}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=pvQtfM4tfb},
}
Figure