Privacy for Rescue: A New Testimony Why Privacy is Vulnerable In Deep Models

Ruiyuan Gao, Ming Dun, Hailong Yang, Zhongzhi Luan, Depei Qian

Introduction: The Split-Inference Privacy Gap

Deep Neural Networks (DNNs) are increasingly deployed in edge-cloud systems to balance computational efficiency with performance. In these "split-inference" models, an edge device (like a smartphone) processes the initial layers of a network, and the resulting intermediate features are transmitted to a cloud server to complete the task.

The prevailing industry assumption is that transmitting intermediate features instead of raw images inherently protects user privacy. Furthermore, researchers often use Mutual Information (MI) between the input and these features as a primary metric for privacy. This paper, "Privacy for Rescue," argues that this is a dangerous misconception. MI is an aggregate measure of "group privacy" that fails to guarantee "pairwise privacy" for individual users. An attacker intercepting these features can often reconstruct sensitive information even when MI scores are low.

Split-inference architecture and the threat model where an attacker intercepts intermediate features.

Redefining Privacy Metrics: PA and PI

To address these vulnerabilities, the authors propose a new framework for evaluating privacy. They define two competing objectives:

  1. User Task ($task_u$): The legitimate classification goal (e.g., identifying an object).
  2. Attacker Task ($task_a$): A malicious goal targeting private information (e.g., identifying a person's presence or attributes).

The researchers introduce a Joint Model approach, assuming a "malicious provider" who trains a specialized model to extract private data from intercepted features. To quantify this risk, they propose two new metrics:

  • Private Accuracy (PA): The ratio of user task accuracy with privacy protection to accuracy without it.
  • Privacy Index (PI): A measure of how much the attacker's accuracy exceeds a random guess. Unlike MI, PI directly reflects the success rate of a targeted privacy breach.

The MI Paradox and the Attacker's Advantage

The study reveals several critical flaws in current privacy-preserving methodologies:

  • The MI Paradox: While the Data Processing Inequality suggests information should decrease deeper in a network, the authors found that calculated MI can actually increase in later layers. This makes MI an unreliable proxy for extractable information.
  • The "Hard Task" Fallacy: Privacy is often overestimated by testing attackers against tasks significantly harder than the user's task. A fair evaluation requires tasks of comparable difficulty.
  • Resilience to Noise: Even when state-of-the-art privacy methods like "Shredder" add noise to features, an attacker can fine-tune their own model on these noisy features to regain high accuracy on the private task.

Key Results: The Illusion of Security

The authors evaluated their metrics across AlexNet, VGG16, and ResNet50 architectures using the ML-Image and VOC2012 datasets. The results highlight a stark contrast between theoretical privacy and actual vulnerability.

Comparison of task accuracy and MI reduction across different model architectures and split points.

In tests with the VGG16 model, the Shredder method achieved a massive 90.54% reduction in MI. Under traditional standards, this would be considered highly secure. However, using the authors' Joint Model approach, the attacker still achieved 71.45% accuracy on the private task (detecting humans). This discrepancy proves that high MI reduction does not equate to effective privacy.

Furthermore, the choice of "cutting point"—where the model is split—is a delicate trade-off. Splitting the model at deeper layers increases the computational burden on the edge device without necessarily providing a proportional increase in privacy protection.

Conclusion and Future Directions

This paper serves as a "testimony" to the research community, urging a shift away from aggregate metrics like MI toward more direct measures of individual privacy risk like PA and PI. Future work should focus on:

  • Dynamic Noise Generation: Developing systems that adapt noise based on specific input features rather than using static distributions.
  • Multi-point Protection: Exploring the addition of privacy-preserving noise at multiple layers simultaneously.
  • Optimization for Privacy: Training future models using PA and PI as explicit loss functions to ensure robust protection against targeted attacks.

Summary updated

Abstract

The huge computation demand of deep learning models and limited computation resources on the edge devices calls for the cooperation between edge device and cloud service by splitting the deep models into two halves. However, transferring the intermediates results from the partial models between edge device and cloud service makes the user privacy vulnerable since the attacker can intercept the intermediate results and extract privacy information from them. Existing research works rely on metrics that are either impractical or insufficient to measure the effectiveness of privacy protection methods in the above scenario, especially from the aspect of a single user. In this paper, we first present a formal definition of the privacy protection problem in the edge-cloud system running DNN models. Then, we analyze the-state-of-the-art methods and point out the drawbacks of their methods, especially the evaluation metrics such as the Mutual Information (MI). In addition, we perform several experiments to demonstrate that although existing methods perform well under MI, they are not effective enough to protect the privacy of a single user. To address the drawbacks of the evaluation metrics, we propose two new metrics that are more accurate to measure the effectiveness of privacy protection methods. Finally, we highlight several potential research directions to encourage future efforts addressing the privacy protection problem.
BibTeX
@misc{gao2019privacy,
  author = {Ruiyuan Gao and Ming Dun and Hailong Yang and Zhongzhi Luan and Depei Qian},
  title = {{Privacy for Rescue: A New Testimony Why Privacy is Vulnerable In Deep Models}},
  year = {2019},
  eprint = {2001.00493},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2001.00493},
}
Figure