Abstract
Machine Learning as a Service (MLaaS) platforms are increasingly vulnerable to model extraction attacks, where adversaries query an API to train a "copycat" model that mimics the original's behavior. This paper introduces BODAME (Bilevel Optimization for Defense Against Model Extraction), a novel framework that protects intellectual property by deploying a surrogate model. By formulating the defense as a bilevel optimization problem, BODAME ensures high utility for legitimate users while intentionally misleading the attacker's training process, causing the extracted model to perform poorly on the defender's target distribution.
Introduction
The rise of MLaaS has made high-quality machine learning models more accessible, but it has also opened the door to intellectual property theft. In a model extraction attack, an attacker sends thousands of queries to a model's API and uses the returned outputs to train their own version of the model. Beyond IP theft, these stolen models can be used to find adversarial vulnerabilities offline.
Existing defenses typically rely on output perturbation—adding noise or rounding decimals—to degrade the quality of the information the attacker receives. However, these methods often degrade performance for legitimate users and can be mitigated by attackers who average out the noise. BODAME shifts the paradigm from simple perturbation to the strategic deployment of a surrogate model designed to deceive the attacker.
The BODAME Framework
BODAME treats the interaction between the defender and the attacker as a game of bilevel optimization. The framework consists of two nested optimization problems:
- The Inner (Lower) Problem: This simulates the attacker's best effort to minimize loss and create a high-fidelity copy of the surrogate model based on their queries.
- The Outer (Upper) Problem: This represents the defender's strategy. The defender seeks a surrogate model $g_\theta$ that remains accurate relative to the "true" model $f$ for legitimate users, but ensures that the attacker’s resulting copycat model $h$ performs poorly on the defender's actual data distribution.
The authors propose two implementations: BODAME-KRR/KR for Kernel Ridge Regression, which can be solved efficiently in polynomial time, and BODAME-SGD/SGA for deep learning, which utilizes hypergradients to optimize the surrogate model against attackers using Stochastic Gradient Descent.
Comparison with Existing Defenses
BODAME distinguishes itself from previous work by being one of the few methods to offer an optimized defense (OptDef) that handles both classification and regression tasks. Unlike simple rounding or detection-based methods, BODAME creates a consistent, deterministic surrogate function that is harder for attackers to bypass.
Experimental Evaluation
The researchers evaluated BODAME on two distinct tasks: a regression task using the UCI White Wine Dataset and a classification task using MNIST.
- Regression (Wine): When the attacker's query distribution was shifted away from the defender's data, the extracted model's Mean Squared Error (MSE) increased significantly, while the defender's surrogate model maintained performance nearly identical to the original true model.
- Classification (MNIST): In scenarios where the attacker queried classes that the defender did not prioritize, the accuracy of the stolen model dropped by approximately 20%. Meanwhile, the surrogate model maintained over 98% accuracy for legitimate users.
- Robustness: Experiments demonstrated that traditional "rounding" defenses had almost no effect on stopping the attacker compared to the BODAME approach.
Key Insights and Limitations
A critical finding of this work is the power of distributional gaps. The defense is most effective when the attacker’s queries come from a different distribution than the defender’s core data. By "bending" the surrogate model’s behavior in out-of-distribution regions, the defender can lead the attacker astray without affecting legitimate performance.
However, the framework currently requires the defender to have some knowledge of the attacker’s model class and training algorithm. Future work aims to develop more "blind" or universal defenses that can operate effectively against a wider variety of unknown attacker architectures.
Summary updated