Protecting Decision Boundary of Machine Learning Model With Differentially Private Perturbation

Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, Jie Shi

Machine Learning as a Service (MLaaS) platforms are increasingly targeted by model extraction attacks, where adversaries attempt to replicate proprietary models by querying their public APIs. This paper introduces Boundary Differential Privacy (BDP), a novel defense that strategically perturbs prediction labels for queries located near a model's decision boundary. By establishing a "privacy buffer" around these sensitive regions, BDP limits the precision with which an attacker can map the model's internal logic while maintaining high accuracy for legitimate users.

The Threat of Model Extraction

MLaaS allows companies to monetize models via APIs. However, these APIs are vulnerable to extraction attacks where an adversary queries the API repeatedly to train a "replica" model that mimics the original. Even when an API only returns a class label (e.g., "Dog" vs. "Cat") instead of a confidence score, attackers can use fine-tuned queries to probe the decision boundary. By identifying the exact points where a label flips, they can reconstruct the model's parameters with high precision.

The interaction between a Machine Learning Service and a Malicious Party attempting model extraction.

Existing defenses, such as monitoring query distributions or rounding confidence scores, often fail against sophisticated attackers or unfairly penalize legitimate users by terminating service early. BDP addresses this by providing a robust privacy guarantee that holds even if an attacker issues an infinite number of queries.

Methodology: The Boundary Differentially Private Layer

The authors propose a "Boundary Differentially Private Layer" (BDPL) that sits between the model and the API output. Its operation involves three main components:

  1. Identifying Sensitive Queries: To preserve utility, the system only adds noise to queries near the decision boundary. Since the boundary's exact shape is often unknown in deep learning, the authors use the Flipping Corner Theorem. They test "corner points" around a query; if any corner has a different label than the query itself, the point is deemed "sensitive."
  2. Perturbation (BRR & MBRR): For sensitive queries, the system applies Boundary Randomized Response (BRR). Instead of returning the true label, it returns a flipped label with a specific probability calculated to satisfy differential privacy ($\epsilon$-BDP). This is generalized to multiple classes (MBRR) using a "counter-class" identification strategy.
  3. Adaptive Defense (ABRR): Recognizing that a "hard" boundary zone might be too rigid, the authors developed a "soft margin" approach. Here, the amount of noise scales continuously based on the query's distance to the boundary, providing smoother protection for complex, non-linear models.
Visualization of decision boundaries and boundary-sensitive zones across different model types.

Key Insights and Results

The core innovation of BDP is precision-based privacy. An attacker does not need the whole feature space; they only need the boundary. BDP guarantees that an adversary cannot learn the boundary beyond a predefined precision. To prevent attackers from "averaging out" the noise, the BDPL caches responses, ensuring the same perturbed label is returned for identical subsequent queries.

Experimental Performance

The researchers evaluated BDP using tabular datasets (Mushrooms, Adult, GSS) and image datasets (MNIST, CIFAR-10). Key findings include:

  • Extraction Resistance: In tabular experiments, BDP reduced the extraction rate (similarity between replica and original) by up to 12% compared to undefended models.
  • High Utility: Most models maintained over 90% accuracy for legitimate users. The "Zone-less" (soft margin) approach was particularly effective for non-linear models.
  • Theoretical Bound: Results show that BDP imposes a "saturation point." After a certain threshold (typically 5K–15K queries), the attacker's ability to improve their replica model plateaus, effectively capping the success of the attack.

Limitations and Future Work

While effective, BDP currently focuses on APIs that return only labels. Future work may explore strategies for APIs that return full probability vectors. Additionally, in very high-dimensional spaces, the "flipping corner" technique requires checking $2d$ points, which could become a computational bottleneck for extremely large input vectors. Finally, the authors suggest exploring "strong adversaries" who are aware of the BDP defense and attempt to use evasion techniques to bypass sensitive zone detection.

Summary updated

Abstract

Machine learning service API allows model owners to monetize proprietary models by offering prediction services to third-party users. However, existing literature shows that model parameters are vulnerable to extraction attacks which accumulate prediction queries and their responses to train a replica model. As countermeasures, researchers have proposed to reduce the rich API output, such as hiding the precise confidence. Nonetheless, even with response being only one bit, an adversary can still exploit fine-tuned queries with differential property to infer the decision boundary of the underlying model. In this article, we propose boundary differential privacy (BDP) against such attacks by obfuscating the prediction responses with noises. BDP guarantees an adversary cannot learn the decision boundary of any two classes by a predefined precision no matter how many queries are issued to the prediction API. We first design a perturbation algorithm called boundary randomized response for a binary model. Then we prove it satisfies <inline-formula><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -BDP, followed by a generalization of this algorithm to a multiclass model. Finally, we generalize a hard boundary to soft boundary and design an adaptive perturbation algorithm that can still work in the latter case. The effectiveness and high utility of our solution are verified by extensive experiments on both linear and non-linear models.
BibTeX
@article{zheng2022protecting,
  author = {Huadi Zheng and Qingqing Ye and Haibo Hu and Chengfang Fang and Jie Shi},
  title = {{Protecting Decision Boundary of Machine Learning Model With Differentially Private Perturbation}},
  journal = {IEEE Trans. Dependable Secur. Comput.},
  year = {2022},
  doi = {10.1109/tdsc.2020.3043382},
}
Figure