TL;DR
This paper introduces "Army of Thieves" (AOT), a novel black-box model extraction technique that leverages an ensemble of diverse "thief" models. By combining the predictions of multiple models, AOT improves sample selection efficiency and enhances the accuracy and robustness of extracted models, outperforming existing methods.
What Problem Does This Paper Address?
Machine learning models deployed as services are vulnerable to "model extraction" or "model stealing" attacks. In these attacks, an adversary queries the deployed model (the "victim") with carefully selected inputs to gather data. This data is then used to train a new model (the "thief") that mimics the victim's functionality. The core challenge lies in efficiently selecting the most informative samples to query, as each query incurs a cost. Existing methods often rely on active learning or semi-supervised learning, but these can be suboptimal in black-box settings, especially when training a single thief model, which might be sensitive to its own capacity or the data it's trained on. This can lead to noisy sample selection and hinder the extraction process.
Methods
The proposed "Army of Thieves" (AOT) attack tackles this by employing an ensemble of five diverse deep learning models as the thief. This ensemble approach aims to leverage "crowd wisdom" to improve sample selection and model extraction.
The pipeline involves two main stages:
- Active Learning for Sample Selection:
- An initial random set of samples is queried from the victim model.
- These samples and their labels are used to train the ensemble of thief models.
- For subsequent query cycles, samples are selected based on the disagreement among the ensemble members. Two strategies are proposed:
- Consensus Entropy: Measures the uncertainty of the ensemble's collective prediction. Samples with higher entropy (more disagreement) are selected.
- Label Disagreement: Directly quantifies the variance in predicted labels across the ensemble. Samples where ensemble members predict different labels are prioritized.
- The selected samples are then queried from the victim, and the process repeats.
- Semi-Supervised Learning (SSL) for Refinement:
- After the active learning query budget is exhausted, the remaining unlabeled data is utilized.
- Inspired by FixMatch, the approach applies weak augmentations to unlabeled samples and uses the ensemble's predictions to generate pseudo-labels.
- Samples with high confidence from the ensemble and consistent predictions across members are selected for further training.
- Strong augmentations are applied to these selected samples before training the ensemble, aiming to improve generalization and robustness.
The ensemble members are chosen to have varying sizes and complexities to ensure diversity, and one member is kept the same architecture as the victim model for comparison.
Datasets
- Training/Evaluation: The paper uses the ImageNet dataset for training and evaluating the thief models. Specifically, the ILSVRC-2012 challenge dataset (1.2 million images) is used as the attack dataset.
- Victim Models: The victim models are trained on four public image classification datasets: CIFAR-10, CIFAR-100, Caltech-256, and CUBS-200.
Key Insights and Ideas
- Ensemble Wisdom for Sample Selection: The core idea is that disagreement among multiple diverse thief models is a strong indicator of informative samples. This is more robust than relying on a single thief model's confidence, which can be biased.
- Mitigating Overfitting: Using an ensemble of models acts as a regularization technique, helping to prevent overfitting, especially when dealing with limited query budgets or complex victim models.
- Improved Robustness and Accuracy: By leveraging collective decisions, the ensemble can produce less noisy and more confident predictions, leading to better extraction accuracy and functional equivalence.
- Handling Black-Box Limitations: The ensemble approach is particularly beneficial in black-box settings where the attacker has limited information about the victim's architecture or training data.
- Adversarial Sample Transferability as a Metric: The paper uses adversarial sample transferability to assess if the thief model has truly captured the victim's decision boundaries, not just its overall accuracy.
Key Results
- Outperformance: The AOT method outperforms previous state-of-the-art methods (ActiveThief, BBD) by at least 3% in accuracy and agreement on CIFAR-10 and CIFAR-100 datasets.
- Adversarial Sample Transferability: AOT achieves a 21% higher adversarial sample transferability than previous work for models trained on CIFAR-10 when the thief model architecture matches the victim.
- Dataset Performance: The method shows strong performance across various datasets, though it notes potential challenges with class imbalance in datasets like CUBS-200, which can affect SSL performance.
- Individual Model vs. Ensemble: While individual strong models like EfficientNet can achieve high accuracy, the ensemble approach generally leads to better overall extraction and robustness.
Limitations and Future Work
- Ensemble Size and Model Choice: The paper acknowledges that computational constraints limited the ensemble size to five and prevented the inclusion of larger models like Vision Transformers (ViT). The selection of appropriate models for the ensemble is crucial.
- Class Imbalance in SSL: The semi-supervised learning stage can be negatively impacted by class imbalance in the selected samples, leading to a decline in overall accuracy, as observed on the CUBS-200 dataset.
- Domain Generalization: While the approach is demonstrated for image classification, the authors suggest it's generic and can be applied to other domains (text, speech) with appropriate active and semi-supervised learning methods.
- Optimizing Ensemble Member Selection: Future work could focus on developing better metrics to quantify the information-capturing ability of ensemble architectures for downstream tasks.
Summary updated