Abstract
This paper proposes a black box extraction attack model on pre-trained image classifiers to rebuild a functionally equivalent model with high similarity. Common model extraction attacks use a large number of training samples to feed the target classifier which is time-consuming with redundancy. The attack results have a high dependency on the selected training samples and the target model. The extracted model may only get part of crucial features because of inappropriate sample selection. To eliminate these uncertainties, we proposed the VAE-kdtree attack model which eliminates the high dependency between selected training samples and the target model. It can not only save redundant computation, but also extract critical boundaries more accurately in image classification. This VAE-kdtree model has shown to achieve around 90% similarity on MNIST and around 80% similarity on MNIST-Fashion with a target Convolutional Network Model and a target Support Vector Machine Model. The performance of this VAE-kdtree model could be further improved by adopting higher dimension space of the kdtree.
Introduction
In adversarial machine learning, model extraction attacks represent a significant security threat where an adversary attempts to "steal" a private, black-box model. By querying the target model and observing its outputs, an attacker can reconstruct a functional clone. However, traditional extraction methods suffer from two primary inefficiencies: they require massive, often redundant datasets to probe the target, and their success is heavily dependent on the similarity between the attacker's data and the original training set.
This research introduces the VAE-kdtree model, a novel framework designed to overcome these limitations. By focusing on identifying decision boundaries rather than replicating entire datasets, the VAE-kdtree approach achieves high similarity with significantly reduced data requirements.
Methodology
The VAE-kdtree architecture integrates two core components: a Variational Autoencoder (VAE) and a k-dimensional (k-d) tree.
- Latent Space Exploration (VAE): The VAE's encoder compresses images into a low-dimensional latent space. By sampling points within this space and passing them through the decoder, the attacker can generate synthetic images that specifically target the decision boundaries of the victim model. This allows for precise "boundary hunting" rather than random sampling.
- Efficient Knowledge Storage (k-d tree): Once synthetic images are labeled by the target model, the resulting vector-label pairs are stored in a k-d tree. This data structure enables efficient K-Nearest Neighbor (KNN) searches during the inference phase of the extracted model.
Experimental Evaluation
The attack was evaluated against two target architectures—a Convolutional Neural Network (CNN) and a Support Vector Machine (SVM)—using the MNIST (handwritten digits) and MNIST-Fashion (clothing items) datasets. The primary metric for success was the similarity between the extracted model's predictions and the target model's predictions.
Results and Discussion
The experimental results demonstrate that the VAE-kdtree model is highly effective across different target architectures. A key finding is the correlation between the latent dimension ($d$) and the attack's precision. As the dimensionality of the latent space increases from 2 to 4, the similarity scores improve significantly, as higher dimensions capture more granular features of the original data.
On the MNIST dataset, the attack achieved over 90% similarity for both CNN and SVM targets at $d=4$. The MNIST-Fashion dataset proved more challenging due to its higher visual complexity, yet the model still achieved approximately 80% similarity. The structural agnosticism of the attack—performing equally well against deep learning and traditional ML models—highlights its versatility as a black-box threat.
Conclusion
The VAE-kdtree model represents a significant advancement in efficient model extraction. By leveraging the generative capabilities of VAEs to probe decision boundaries and the search efficiency of k-d trees, the attack minimizes data redundancy while maintaining high fidelity. Future work will focus on scaling this approach to more complex datasets like ImageNet, where the current limitations of VAE-generated image clarity present a challenge for capturing fine-grained classification features.
Summary updated