When a company deploys a state-of-the-art Large Language Model (LLM) or an embedding service, they are exposing a multi-million dollar asset through a narrow window: the API. While the weights remain hidden, the model's behavior is public. This exposure creates a vulnerability known as model extraction, where an adversary queries the API to train a "student" model that mimics the proprietary "teacher."
As Kamalika Chaudhuri and colleagues demonstrated in Model Extraction and Active Learning, stealing a model is often a specialized form of active learning. By strategically picking queries, an attacker can reconstruct a functional replica for a fraction of the original training cost. This has led to a surge in research focused on protecting machine learning intellectual property (IP).
The Mechanism of Theft: Knowledge Distillation
To understand the defense, we must first understand the attack. Most extraction attacks rely on Knowledge Distillation. In a standard setting, distillation is a feature: it allows a small model to learn from a large one by matching the teacher's "soft targets"—the full probability distribution over possible outputs.
However, in an extraction attack, this becomes a bug. Haoyu Ma and others note in Undistillable: Making A Nasty Teacher That CANNOT teach students that an adversary doesn't need your data or your weights; they only need to observe how your model reacts to specific inputs. If the student model can minimize the difference between its predictions and yours, it effectively "absorbs" your model's logic.
Inheritable Watermarks: Marking the Stolen Goods
If you cannot prevent a model from being distilled, the next best option is to ensure the stolen copy carries a "brand." This is the intuition behind inheritable watermarking.
Traditional watermarking might involve a "trigger set"—specific inputs that produce a hard-coded, unusual output. But these are easily stripped. Modern techniques like Entangled Watermarking Embeddings (EWE), proposed by Christopher A. Choquette-Choo in Entangled Watermarks as a Defense against Model Extraction, "entangle" the watermark with the model's primary task. Because the watermark is embedded in the same features the model uses for its main predictions, a student model cannot learn the task without also learning the watermark.
For generative models, Xuandong Zhao introduced GINSEW in Protecting Language Generation Models via Invisible Watermarking. This method embeds a hidden sinusoidal signal into the output probability distributions. When a student model trains on these distributions, it unknowingly optimizes itself to replicate that signal, providing a mathematical "smoking gun" for ownership verification.
Nasty Teachers: Sabotaging the Distillation Process
A more aggressive defense is to make the model "undistillable." The goal is to create a Nasty Teacher: a model that yields high accuracy for legitimate users but provides misleading information to anyone trying to use its outputs for training.
The core idea, explored by Haoyu Ma, is self-undermining knowledge distillation. By slightly perturbing the "soft labels" (the probabilities) while keeping the "hard label" (the top prediction) correct, the teacher can confuse the student's gradient descent. The student model sees a landscape of information that looks like valid knowledge but actually leads to poor generalization.
However, this is a cat-and-mouse game. Souvik Kundu showed in Analyzing the Confidentiality of Undistillable Teachers in Knowledge Distillation that a "Skeptical Student" can bypass these defenses by focusing on the teacher's intermediate representations rather than just the final output, suggesting that defenses must be deep-seated to be effective.
Stateful Detection: Identifying the Thief
While watermarking and poisoning focus on the model's output, stateful detection focuses on the user's behavior. An attacker's query pattern is rarely "natural." To extract a model efficiently, they must query the decision boundaries or cover a wide range of the input space.
Soham Pal developed VarDetect in Stateful Detection of Model Extraction Attacks, which uses a Variational Autoencoder to monitor the distribution of queries over time. If a user's queries begin to look like they are systematically "mapping" the model rather than using it for its intended purpose, the system can flag the account or inject noise into the responses. Similarly, for Graph Neural Networks, Jing Xu proposed ADAGE in ADAGE: Active Defenses Against GNN Extraction, which detects attackers by monitoring how many distinct graph communities their queries attempt to cover.
The Path Forward: Dataset Inference
When watermarks fail or are removed, owners can turn to Dataset Inference (DI). As Pratyush Maini and Nicolas Papernot detailed in Dataset Inference: Ownership Resolution in Machine Learning, DI doesn't require modifying the model at all. Instead, it probes a suspect model to see if it possesses "private knowledge" of the original training set. Because models naturally "memorize" their training data to some extent, a stolen model will react differently to samples from the original dataset than to unseen data, allowing for high-confidence ownership claims without any prior defensive measures.