Learning Model with Error - Exposing the Hidden Model of BAYHENN

Harry W. H. Wong, Jack P. K. Ma, Donald P. H. Wong, Lucien K. L. Ng, Sherman S. M. ChowIJCAI 2020 · DOI

TL;DR

This paper demonstrates that a recently proposed privacy-preserving deep neural network (DNN) inference technique called BAYHENN is not as secure as claimed. The authors present a practical attack that can recover the underlying DNN model by exploiting how BAYHENN uses noise, effectively undermining its claimed security based on the Learning With Errors (LWE) assumption.

What Problem Does This Paper Address?

The paper tackles the critical challenge of privacy in machine learning as a service (MLaaS). When users submit sensitive data to a MLaaS platform for inference by a deep neural network (DNN), there's a risk of both the data being exposed and the valuable trained model being leaked. While homomorphic encryption (HE) offers a way to perform computations on encrypted data, it struggles with non-linear operations common in DNNs. BAYHENN proposed a solution by using a Bayesian Neural Network (BNN) with noisy weights, claiming that clients could get intermediate results without learning the exact model parameters, relying on the Learning With Errors (LWE) assumption for security. This paper investigates and refutes this security claim, showing that the noise introduced in BAYHENN is not sufficient to protect the model.

Methods

The core of this paper is a theoretical and empirical analysis that refutes BAYHENN's security claims. The authors:

  • Formalize a Security Game: They define a security game with specific oracle queries that capture realistic threat models for an attacker.
  • Develop a Semi-Honest Attack: They devise an attack that, under a semi-honest adversary model, can recover all parameters of a single-layer BAYHENN. This attack leverages the fact that BAYHENN's noisy intermediate results can be used to reconstruct the model parameters.
  • Generalize to Malicious Attacks: The attack is extended to recover the entire multi-layer model, even under a malicious adversary model or with an increased number of queries.
  • Analyze LWE Misuse: They argue that BAYHENN misuses the LWE assumption by not sampling the necessary "basis" uniformly at random, which is crucial for LWE's security. This allows their attack to effectively estimate the model parameters rather than solving a hard cryptographic problem.

Datasets

The paper mentions using the MNIST dataset for their experiments to evaluate the effectiveness of their attack on a single-layer BAYHENN model. The test dataset for evaluation consists of 10,000 samples.

Key Insights and Ideas

  • Noise is Not Always Security: The paper highlights that simply adding noise (as in BAYHENN's BNN approach) does not automatically provide cryptographic security. The type and distribution of noise, and how it interacts with the underlying cryptographic assumptions (like LWE), are crucial.
  • LWE Assumption Vulnerability: BAYHENN's reliance on LWE is shown to be flawed because the "basis" (the input to the model) is not sampled uniformly at random, a key requirement for LWE's hardness. This makes the problem tractable for an attacker.
  • Practical Attack Feasibility: The authors demonstrate that recovering the model is not just theoretically possible but practically achievable with a reasonable number of queries, even for a semi-honest adversary.
  • Need for Interdisciplinary Collaboration: The paper emphasizes the importance of collaboration between cryptographers and machine learning experts to develop truly secure and practical privacy-preserving solutions.

Key Results

  • The paper successfully refutes the security claim of BAYHENN.
  • An attack is presented that can recover the parameters of a single-layer BAYHENN model.
  • This attack can be generalized to recover the entire multi-layer model.
  • The extracted model is shown to be "as good as" the BNN approximation of the original DNN.
  • In experiments, with a sufficient number of queries (m) and a large sample size (S=512), the extracted model achieved 98.22% matching results compared to the ground truth model.

Limitations and Future Work

  • Limitations: The paper focuses on single-layer attacks and then generalizes to multi-layer recovery. It acknowledges that more complex neural network architectures (like CNNs) might require further investigation. The authors also note that while BAYHENN assumes semi-honest attackers, malicious adversaries are a relevant concern.
  • Future Work: The authors pose open problems, including:
  • Investigating the possibility of complete model extraction under the semi-honest model.
  • Exploring model extraction for more complex neural networks (e.g., CNNs with hidden hyperparameters) within the BAYHENN framework.

Summary updated

BibTeX
@inproceedings{wong2020learning,
  author = {Harry W. H. Wong and Jack P. K. Ma and Donald P. H. Wong and Lucien K. L. Ng and Sherman S. M. Chow},
  title = {{Learning Model with Error - Exposing the Hidden Model of BAYHENN}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/488},
}
Figure