Abstract
In the context of prediction-as-a-service, concerns about the privacy of the data and the model have been brought up and tackled via secure inference protocols. These protocols are built up by using single or multiple cryptographic tools designed under a variety of different security assumptions. In this paper, we introduce SECO, a secure inference protocol that enables a user holding an input data vector and multiple server nodes deployed with a split neural network model to collaboratively compute the prediction, without compromising either party's data privacy. We extend prior work on secure inference that requires the entire neural network model to be located on a single server node, to a multi-server hierarchy, where the user communicates to a gateway server node, which in turn communicates to remote server nodes. The inference task is split across the server nodes and must be performed over an encrypted copy of the data vector. We adopt multiparty homomorphic encryption and multiparty garbled circuit schemes, making the system secure against dishonest majority of semi-honest servers as well as protecting the partial model structure from the user. We evaluate SECO on multiple models, achieving the reduction of computation and communication cost for the user, making the protocol applicable to user's devices with limited resources.
Introduction
As "Prediction-as-a-Service" (PaaS) becomes ubiquitous, two major privacy hurdles remain: data privacy and model privacy. Users are increasingly hesitant to share sensitive raw data, such as medical records or financial history, with cloud providers. Simultaneously, companies seek to protect their proprietary model weights and architectures, which often represent significant intellectual property.
Existing state-of-the-art protocols like DELPHI or GAZELLE address these concerns in single-server environments. However, they struggle with modern large-scale AI models that require distributed GPU clusters. Furthermore, these protocols often demand that the user's device remains online and performs heavy computations for every layer of the network, which both leaks architectural details and drains the battery of mobile devices.
The SECO Architecture
SECO (Secure Inference with Model Splitting) addresses these challenges by introducing a multi-server hierarchy. Instead of a single server, the model is split across a Gateway Server (Server A) and multiple Remote Servers (Servers B and C).
In this hierarchy, the user only communicates directly with the Gateway Server. The neural network model is divided: the initial layers reside on the Gateway, while the subsequent layers are secret-shared and distributed among the Remote Servers. This structure allows the user to participate only in the evaluation of the initial layers. Once these are complete, the user can effectively go offline while the remote servers finalize the computation.
Protocol Phases and Cryptographic Foundations
The SECO protocol operates in three distinct phases: Setup, Preprocessing, and Inference. It leverages a hybrid of Multiparty BFV Homomorphic Encryption (MPHE) and Garbled Circuits (GC).
- Setup: Involves MPHE key generation and the preprocessing of the remote model components ($F_{BC}$).
- Preprocessing: The user and the Gateway Server perform preprocessing for the initial model layers ($F_A$).
- Inference: The actual computation occurs. A critical innovation in SECO is the "Secure Transition" sub-protocol, which moves intermediate results from the user-gateway pair to the remote server cluster without leaking user data or model structure.
Unlike previous works where the user often acts as the evaluator for Garbled Circuits, SECO assigns these roles to the remote servers (e.g., Server C evaluates while Server B garbles), drastically reducing the user's computational and communication overhead.
Evaluation and Results
The authors evaluated SECO using standard benchmarks: ResNet32 (a 62-layer deep network), MiniONN, and LeNet. The results demonstrate that SECO significantly improves the end-user experience:
- Communication Efficiency: User-side communication costs are reduced by up to 18.5x compared to the DELPHI protocol.
- Time Savings: User participation time is slashed by up to 9.9x. Depending on the layer split, the online speed for the user is 2.6x to 11x faster.
- Hardware Accessibility: By offloading the bulk of the work to the server hierarchy, SECO can run on devices with as little as 2 GB of memory, such as smartwatches, which would otherwise fail to support large-scale secure inference.
Conclusion and Future Work
SECO represents a significant step toward making secure inference practical for resource-constrained devices. By shifting the computational burden to a multi-server hierarchy, it protects model privacy while ensuring user efficiency.
While currently optimized for Convolutional Neural Networks (CNNs) under a semi-honest security assumption, future work includes extending the protocol to handle malicious adversaries and Transformer-based architectures common in Large Language Models (LLMs). The trade-off for this user-side efficiency is increased server-to-server communication, a necessary compromise for modern mobile-first AI applications.
Summary updated