Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile Computing

Mohammad Malekzadeh, Fahim Kawsar

Abstract

In split inference, a deep neural network (DNN) is partitioned to run the early part of the DNN at the edge and the later part of the DNN in the cloud. This meets two key requirements for on-device machine learning: input privacy and computation efficiency. Still, an open question in split inference is output privacy, given that the outputs of the DNN are observable in the cloud. While encrypted computing can protect output privacy too, homomorphic encryption requires substantial computation and communication resources from both edge and cloud devices. In this paper, we introduce Salted DNNs: a novel approach that enables clients at the edge, who run the early part of the DNN, to control the semantic interpretation of the DNN's outputs at inference time. Our proposed Salted DNNs maintain classification accuracy and computation efficiency very close to the standard DNN counterparts. Experimental evaluations conducted on both images and wearable sensor data demonstrate that Salted DNNs attain classification accuracy very close to standard DNNs, particularly when the Salted Layer is positioned within the early part to meet the requirements of split inference. Our approach is general and can be applied to various types of DNNs. As a benchmark for future studies, we open-source our code.

Introduction

In the landscape of mobile AI, split inference has emerged as a vital strategy to balance the limited resources of edge devices with the high performance of cloud-based deep neural networks (DNNs). By partitioning a model, the "early part" runs locally on a mobile device to protect raw input data, while the "later part" is offloaded to a powerful server. However, a significant privacy gap remains: output privacy. Even if the server never sees the raw input, it observes the final classification results, which can reveal sensitive information about a user's health, location, or behavior.

Current solutions like homomorphic encryption are often too computationally expensive for real-time mobile applications. This paper introduces Salted DNNs, a novel approach that allows clients to scramble the semantic meaning of a model's final predictions using a secret "salt" value, effectively achieving output privacy with minimal impact on performance.

Methodology: The Salted DNN Architecture

The core innovation of Salted Inference is making the model's output "language" dependent on a secret key, or salt, known only to the client. This ensures that while the server performs the heavy lifting of computation, it cannot interpret the final results.

Overview of the Salted DNN architecture for split inference, showing how the salt $s$ permutes the output semantics.

The process involves four key stages:

  1. Salt Input: The client selects a secret salt $s$ (a random integer). This salt acts as a key that determines how the output classes are permuted.
  2. The Salted Layer: To integrate this salt, the authors use a transposed convolution layer. This layer expands the simple salt value into a feature map that is concatenated with the network's internal data.
  3. Training for Flexibility: During training, the model is taught to produce different permuted outputs corresponding to various salts. The network essentially learns a multi-task mapping: classifying data while simultaneously rearranging those classes based on the salt.
  4. Inference and Decoding: At runtime, the client processes the early layers with their secret salt and sends the intermediate result to the server. The server produces a "salted" output, which it cannot interpret. The client receives this scrambled result and uses their secret salt to map it back to the correct category.

Experimental Evaluation

The researchers evaluated Salted DNNs on two distinct datasets to prove versatility: CIFAR10 (image recognition) and PAMAP2 (wearable sensor data for activity recognition). These represent common use cases for mobile AI in computer vision and IoT health monitoring.

Accuracy and Salt Position

A critical finding was the impact of the "Salted Layer" position on model performance. Placing the salt in the early layers—a requirement for split inference—maintains high accuracy while forcing the salt to pass through more non-linear layers, making it harder for a server to reverse-engineer.

Classification accuracy of Salted DNNs across different datasets, architectures, and salt positions compared to a "No Salt" baseline.

As shown in the table above, Salted DNNs achieved performance remarkably close to standard models. On CIFAR10 using WideResNet, the Salted DNN reached 87.2% accuracy, compared to 88.3% for the standard model. On the PAMAP2 sensor dataset, it reached 89.8%, compared to a 92.1% baseline. These results indicate that the privacy gains are achieved with very high fidelity to the original model's performance.

Discussion and Conclusion

Salted Inference provides a computationally efficient alternative to encrypted computing. By adding only a lightweight transposed convolution layer (adding only 16K parameters to a 3.2M parameter WideResNet model), the approach introduces negligible inference overhead for mobile devices.

While highly effective for classification tasks with a moderate number of classes, challenges remain for high-cardinality datasets where accuracy degradation becomes more pronounced. Future work will focus on scaling the architecture for more complex tasks like regression and object detection, as well as establishing formal mathematical privacy guarantees against sophisticated adversarial attacks.

Summary updated

Abstract

In split inference, a deep neural network (DNN) is partitioned to run the early part of the DNN at the edge and the later part of the DNN in the cloud. This meets two key requirements for on-device machine learning: input privacy and computation efficiency. Still, an open question in split inference is output privacy, given that the outputs of the DNN are observable in the cloud. While encrypted computing can protect output privacy too, homomorphic encryption requires substantial computation and communication resources from both edge and cloud devices. In this paper, we introduce Salted DNNs: a novel approach that enables clients at the edge, who run the early part of the DNN, to control the semantic interpretation of the DNN's outputs at inference time. Our proposed Salted DNNs maintain classification accuracy and computation efficiency very close to the standard DNN counterparts. Experimental evaluations conducted on both images and wearable sensor data demonstrate that Salted DNNs attain classification accuracy very close to standard DNNs, particularly when the Salted Layer is positioned within the early part to meet the requirements of split inference. Our approach is general and can be applied to various types of DNNs. As a benchmark for future studies, we open-source our code.
BibTeX
@inproceedings{malekzadeh2024salted,
  author = {Mohammad Malekzadeh and Fahim Kawsar},
  title = {{Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile Computing}},
  booktitle = {HotMobile},
  year = {2024},
  doi = {10.1145/3638550.3641131},
}
Figure