Abstract
Deep-learning-as-a-service is a novel and promising computing paradigm aiming at providing machine/deep learning solutions and mechanisms through Cloud-based computing infrastructures. Thanks to its ability to remotely execute and train deep learning models (that typically require high computational loads and memory occupation), such an approach guarantees high performance, scalability, and availability. Unfortunately, such an approach requires to send information to be processed (e.g., signals, images, positions, sounds, videos) to the Cloud, hence having potentially catastrophic-impacts on the privacy of users. This paper introduces a novel distributed architecture for deep-learning-as-a-service that is able to preserve the user sensitive data while providing Cloud-based machine and deep learning services. The proposed architecture, which relies on Homomorphic Encryption that is able to perform operations on encrypted data, has been tailored for Convolutional Neural Networks (CNNs) in the domain of image analysis and implemented through a client-server REST-based approach. Experimental results show the effectiveness of the proposed architecture.
Introduction
The "Deep-Learning-as-a-Service" (DLaaS) paradigm enables users to leverage cloud infrastructure for complex AI tasks without local hardware constraints. However, the requirement to transmit raw, sensitive data—such as medical records or personal photos—to cloud providers creates a significant privacy barrier. Homomorphic Encryption (HE) offers a solution by allowing computations directly on encrypted data. However, HE is computationally intensive and restricted to basic polynomial operations (addition and multiplication), making standard neural network components like ReLU activations and Max-Pooling incompatible out-of-the-box.
Methodology: The HE-DL Architecture
The proposed HE-DL architecture utilizes the HE-BFV encryption scheme to bridge the gap between cloud efficiency and user privacy. The system operates through a client-server model where the client handles encryption and decryption, while the cloud performs the heavy lifting on encrypted ciphertexts.
Model Approximation and Encoding
To achieve HE compatibility, standard CNNs must be approximated to use only polynomial operations:
- Activations: ReLU functions are replaced with a square function ($x^2$).
- Pooling: Max-Pooling is replaced with Average-Pooling (addition and multiplication by a fraction).
The model is retrained on plaintext data using these approximations so weights can adjust before being encoded for HE.
Operating Modalities
- Recall: The cloud executes the entire network to return a final encrypted classification label.
- Transfer Learning: The cloud processes only the initial layers to extract features. The client decrypts these features to train a local, lightweight classifier (e.g., SVM), preserving the "noise budget" of the HE scheme and reducing cloud compute time.
Experimental Results
The system was evaluated using the FashionMNIST and MNIST datasets. Implementation utilized the PyCrCNN library (wrapping Microsoft SEAL) on AWS EC2.
Accuracy and Parameter Trade-offs
A critical finding was the impact of HE parameters—specifically the polynomial modulus degree $m$ and plaintext modulus $p$—on model performance. If parameters are too low, the "noise" inherent in HE operations corrupts the data, leading to random-guess accuracy. With optimal parameters, the encrypted models achieved accuracy identical to their plaintext approximated counterparts.
Interestingly, the 5-layer CNN outperformed the 6-layer CNN after approximation, despite the 6-layer model being superior in standard plaintext form. This highlights that HE-compatible architectures require specific design rather than simple adaptation of existing models.
Performance Metrics
- Recall Latency: A 6-layer CNN required ~11.8s of server time and ~2.2s of client time per image.
- Transfer Learning Latency: Processing only the first four layers reduced server time to ~5.5s.
- Network Overhead: Data expansion from HE resulted in transmission times between 2 and 14.7 seconds depending on parameters.
Conclusion
HE-DL demonstrates a viable path for privacy-preserving DLaaS. While HE introduces significant computational and network overhead, the use of model approximation and transfer learning strategies makes it feasible for sensitive applications. Future work will focus on automating HE parameter configuration and extending the architecture to Recurrent Neural Networks (RNNs) and resource-constrained IoT devices.
Summary updated