Marich: A Query-efficient & Online Model Extraction Attack using Public Data

Pratik Karmakar, Debabrota BasuICLR 2023 · OpenReview

TL;DR

This paper introduces MARICH, a novel query-efficient black-box model extraction attack that uses active learning to create an informative replica of a target ML model with minimal queries. MARICH achieves high accuracy and demonstrates that extracted models can be used for privacy attacks like membership inference, highlighting the security implications of publicly accessible ML models.

What Problem Does This Paper Address?

Machine Learning as a Service (MLaaS) platforms offer ML models via APIs, making them accessible but also vulnerable to "black-box" model extraction attacks. These attacks aim to create a functional replica of the target model by querying its API. A key challenge is the trade-off between the number of queries (which incurs cost and risk of detection) and the accuracy of the extracted model. Existing methods often require a large number of queries or specific feedback (like probability distributions), which aren't always available through public APIs. This paper addresses the need for a query-efficient attack that can operate with minimal, publicly available information and still produce a high-fidelity replica.

Methods

The core of MARICH is an active learning strategy designed to select the most informative queries. The authors frame the problem as an online variational optimization task, aiming to maximize the mutual information between the target and extracted models' predictions while minimizing the difference in their output distributions. MARICH employs a three-phase query selection process:

  1. Initialization: A small set of initial queries are randomly selected to train a preliminary extracted model.
  2. Adaptive Phase: This phase iteratively selects queries using three strategies:
  3. Entropy Sampling: Selects queries where the extracted model is most uncertain about its predictions, aiming to improve its overall accuracy.
  4. Entropy Gradient Sampling: Refines the selection by considering the gradients of entropy, promoting diversity among selected queries.
  5. Loss Sampling: Identifies queries where the target and extracted models have the largest prediction mismatch, forcing the extracted model to better align with the target.

This adaptive approach allows MARICH to efficiently gather information and progressively improve the extracted model.

Datasets

The authors used a variety of datasets for both training target models and as public query datasets for the extraction attacks:

  • Training Datasets (for target models): MNIST, CIFAR10, BBC News.
  • Query Datasets (publicly available and distinct from training data): EMNIST, CIFAR10, STL10, AGNews.

Key Insights and Ideas

  • Distributional Equivalence: The paper introduces a new notion of "distributional equivalence" for model extraction, which is more robust than simply matching predictions on a fixed dataset. It focuses on matching the joint distributions of inputs and outputs.
  • Max-Information Extraction: A complementary objective is proposed: maximizing the mutual information between the target and extracted models' output distributions, aiming for an "informative replica."
  • Query-Efficient Active Learning: MARICH's multi-stage sampling strategy (Entropy, Entropy Gradient, Loss) is crucial for balancing query efficiency with the goal of creating a high-fidelity and informative model replica.
  • Privacy Implications: Extracted models are shown to be effective for membership inference attacks, demonstrating that successful model extraction can directly lead to privacy breaches of the original training data.
  • Model Mismatch Handling: MARICH can handle cases where the extracted model has a different architecture or complexity than the target model, though this can impact accuracy.

Key Results

  • Accuracy: MARICH successfully extracts models that achieve a significant portion of the target model's accuracy, ranging from 68.22% to 98.52% across various models (LR, ResNet18, BERT) and datasets. For example, it extracted a logistic regression model with 90.69% of the target's accuracy using only 3.5% of the query dataset.
  • Query Efficiency: The attack is highly query-efficient, using only 0.83% to 6.15% of the query dataset samples (e.g., 1,070 queries for a BERT model). This is significantly fewer queries than other active learning attacks.
  • Informativeness (Membership Inference): Extracted models are highly informative for membership inference attacks, showing agreement rates with the target model's MI capabilities of 75.52% to 93.97%. This indicates the extracted models are not just accurate but also retain sensitive information about the training data.
  • Performance against DP: MARICH's performance degrades slightly (2.3-7.4% lower accuracy) when attacking differentially private target models, but it remains effective.

Limitations and Future Work

  • Model Mismatch Impact: While MARICH can handle model mismatch, using a less complex model to extract a more complex one leads to a significant drop in accuracy. The authors suggest this is due to the limited representation capacity of simpler models.
  • Hyperparameter Tuning: The performance of MARICH is sensitive to hyperparameters like initial sample size and budget, requiring careful tuning (as demonstrated in Section F).
  • Future Work: The paper poses an interesting question about whether distributional equivalence extraction is "easier" than functional equivalence extraction, which is known to be NP-hard. Further research could explore this theoretical aspect and investigate more advanced defense mechanisms against such attacks.

Summary updated

Abstract

In this paper, we study black-box model stealing attacks where the attacker is only able to query a machine learning model through publicly available APIs. Specifically, our aim is to design a black-box model stealing attack that uses a minimal number of queries to create an informative replica of the target model. First, we reduce this problem to an online variational optimization problem. At every step, the attacker solves this problem to select the most informative query that maximizes the entropy of the selected queries and simultaneously reduces the mismatch between the target and the stolen models. We propose an online and adaptive algorithm, Marich, that leverages active learning to select the queries. We instantiate efficiency of our attack against different models, including logistic regression, BERT and ResNet18, trained on different text and image datasets. Marich is able to steal a model that can achieve 70-96$\%$ of true model's accuracy using 0.8-10$\%$ samples from the attack datasets which are publicly available and different from the training datasets. Our stolen models also achieve 75-98$\%$ accuracy of membership inference and also show 70-90$\%$ agreement of membership inference with direct membership inference on the target models. Our experiments validate that Marich is query-efficient and capable of creating an informative replica of the target model.
BibTeX
@inproceedings{karmakar2023marich,
  author = {Pratik Karmakar and Debabrota Basu},
  title = {{Marich: A Query-efficient \& Online Model Extraction Attack using Public Data}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=kocBczDfBeT},
}
Figure