Inference Privacy: Properties and Mechanisms

Fengwei Tian, Ravi Tandon

TL;DR

This paper introduces "Inference Privacy" (IP), a new framework to protect user data during the inference phase of machine learning models, addressing a gap in existing privacy-preserving techniques. They propose novel input and output perturbation mechanisms and demonstrate through experiments that while higher privacy guarantees often lead to reduced model utility, IP offers a flexible way to balance these trade-offs.

What Problem Does This Paper Address?

Machine learning models are increasingly used in applications that require continuous interaction with users, raising significant privacy concerns. While much research has focused on protecting data during the training phase (e.g., preventing membership inference attacks), protecting user data during the inference phase (when a user queries a trained model) has received less attention. However, it's possible for adversaries to infer sensitive user information from model outputs even after training. This paper aims to bridge this gap by formalizing and enabling privacy guarantees specifically for the inference stage.

Methods

The core of the paper is the formalization of Inference Privacy (IP). IP is defined such that if two inputs are close to each other within a certain radius ($\alpha$), their corresponding outputs from a privacy mechanism should be statistically similar. This similarity is quantified by privacy parameters ($\epsilon$, $\delta$).

The authors propose two main categories of mechanisms to achieve IP:

  • Output Perturbation: This involves adding noise directly to the model's output. The amount of noise is determined by the model's sensitivity, specifically its Lipschitz constant. Two variants are presented:
  • Lap-Output Mechanism: Uses the $\ell_1$ global Lipschitz constant and injects Laplace noise.
  • Gauss-Output Mechanism: Uses the $\ell_2$ global Lipschitz constant and injects Gaussian noise, requiring a privacy loss parameter $\delta$.
  • Input Perturbation: This method adds noise to the input before it's fed into the model.
  • Gauss-Input Mechanism: Injects Gaussian noise into the input. This approach is more universal as it doesn't depend on the specific model's architecture or sensitivity.

The paper also explores theoretical properties of IP, including its resilience to post-processing, and how multiple IP mechanisms can be composed (both sequentially and in parallel) while maintaining an overall privacy guarantee.

Datasets

The experiments were conducted on two standard image classification datasets:

  • CIFAR-10
  • CIFAR-100

Key Insights and Ideas

  • IP as a Generalization: Inference Privacy (IP) is shown to be a generalization of Local Differential Privacy (LDP), offering a more flexible framework by incorporating a distance metric and radius ($\alpha$) for input similarity.
  • Trade-off is Inevitable: A fundamental insight is the inherent trade-off between privacy and utility. As privacy requirements (larger $\alpha$, smaller $\epsilon$) increase, model accuracy (utility) generally decreases.
  • Mechanism Choice Matters: The choice of mechanism and model architecture significantly impacts the utility-privacy trade-off. For instance, Gauss-Output on specialized Lipschitz-constrained models (SLL) often performed better in high-privacy regions than Gauss-Input on standard models.
  • Fine-tuning Boosts Utility: Fine-tuning pre-trained models (like ResNet-18) with noisy inputs (as used in Gauss-Input) can significantly improve utility compared to using the original pre-trained model.
  • Lipschitz Constant is Key (and a Challenge): Output perturbation heavily relies on the model's Lipschitz constant. While smaller Lipschitz constants (like in SLL models) are beneficial, estimating them accurately for large, complex models (like standard ResNets) is challenging, making input perturbation a more universally applicable alternative.
  • Chaining Property: The IP framework allows for a "chaining property," meaning privacy guarantees can be extended across different scenarios or input distances, offering flexibility.

Key Results

  • Radius Impact: Increasing the privacy radius ($\alpha$) consistently decreased classification accuracy across various mechanisms and models, as shown in Figure 3.
  • Privacy Budget Impact: Increasing the privacy budget ($\epsilon$) also led to decreased accuracy, demonstrating the direct relationship between privacy strength and utility loss (Figure 4).
  • Mechanism Performance:
    • On CIFAR-10 and CIFAR-100, the Gauss-Output mechanism applied to SLL models generally outperformed the Gauss-Input mechanism on both SLL and fine-tuned ResNet-18 models, especially in high-privacy regions (low $\epsilon$, larger $\alpha$).
  • However, in low-privacy regions (high $\epsilon$, smaller $\alpha$), the Gauss-Input mechanism on fine-tuned ResNet-18 showed competitive performance.
  • Table 6 Summary: The table provides a detailed comparison of natural accuracy under various IP constraints. For example, on CIFAR-10 with a Gauss-Input mechanism on a fine-tuned ResNet-18, accuracy dropped from 93.11% (radius 0.001, $\epsilon=0.1$) to 10.16% (radius 0.1, $\epsilon=0.1$).

Limitations and Future Work

  • Lipschitz Constant Estimation: Accurately estimating Lipschitz constants for large neural networks is computationally challenging, which limits the direct applicability of output perturbation methods.
  • Universal Applicability: While input perturbation is more universal, it might incur higher utility loss compared to tailored output perturbation methods.
  • Future Directions:
    • Investigating local Lipschitz constants instead of global ones for designing IP mechanisms to potentially improve the utility-privacy trade-off.
  • Exploring IP for other domains beyond image classification, such as natural language processing, leveraging its flexibility in defining metrics and radii.
  • Leveraging the post-processing property with hybrid models that incorporate 1-Lipschitz components to achieve better trade-offs.

Summary updated

Abstract

Ensuring privacy during inference stage is crucial to prevent malicious third parties from reconstructing users' private inputs from outputs of public models. Despite a large body of literature on privacy preserving learning (which ensures privacy of training data), there is no existing systematic framework to ensure the privacy of users' data during inference. Motivated by this problem, we introduce the notion of Inference Privacy (IP), which can allow a user to interact with a model (for instance, a classifier, or an AI-assisted chat-bot) while providing a rigorous privacy guarantee for the users' data at inference. We establish fundamental properties of the IP privacy notion and also contrast it with the notion of Local Differential Privacy (LDP). We then present two types of mechanisms for achieving IP: namely, input perturbations and output perturbations which are customizable by the users and can allow them to navigate the trade-off between utility and privacy. We also demonstrate the usefulness of our framework via experiments and highlight the resulting trade-offs between utility and privacy during inference.
BibTeX
@inproceedings{tian2024inference,
  author = {Fengwei Tian and Ravi Tandon},
  title = {{Inference Privacy: Properties and Mechanisms}},
  booktitle = {IEEECONF},
  year = {2024},
}
Figure