As machine learning models become high-value intellectual property, the risk of "model theft"—where an adversary steals a model's functionality through extraction or uses proprietary data without authorization—has grown significantly. Because these models are often served behind black-box APIs, owners cannot simply inspect the weights to prove theft. Instead, a new research direction has emerged that treats ownership verification as a problem of statistical hypothesis testing.
By analyzing the output distributions of a suspicious model, researchers can calculate the mathematical probability that its behavior is a result of independent training versus unauthorized derivation.
The Statistical Framework for Provenance
The core intuition behind this field is that a model's behavior is never truly random; it carries a "signal" from its training process. Verification methods attempt to reject a null hypothesis: the assumption that the suspicious model was trained independently from scratch.
To reject this hypothesis, practitioners look for statistical anomalies that are astronomically unlikely to occur by chance. For example, Kallas et al. argue that by quantifying ownership through statistical rarity, one can create proofs of intellectual property that exceed one hundred bits of security. Similarly, Ruibo Chen et al. propose treating model correlation as a hypothesis test, using the transferability of "nonsense" triggers to generate rigorous $p$-values that distinguish derivative works from coincidental similarities.
Watermarking as a Statistical Signal
The most common way to embed these signals is through watermarking, where a model is intentionally trained to exhibit specific behaviors on "trigger" inputs.
- Backdoor Triggers: Hongsheng Hu demonstrates that by "tagging" data with backdoors, owners can later query a model and use a t-test to determine if the model's response to the trigger is a result of training on that specific data.
- Margin-Based Stickiness: To ensure watermarks survive model extraction, Byungjoo Kim et al. maximize the decision margins of trigger samples. This makes the watermark "sticky," forcing a stolen model to inherit the behavior with high statistical significance.
- Linguistic Patterns: In natural language generation, Xu et al. and He et al. use secret lexical swaps or conditional linguistic patterns. By calculating the probability that a competitor's model would choose the exact same synonym variations by chance, they provide mathematically grounded proof of theft.
Auditing Without Active Injection
In many cases, a defender cannot inject a watermark before a model is stolen. Researchers have therefore developed "non-invasive" auditing techniques that rely on the inherent statistical properties of the model or dataset.
- Distributional Alignment: Gao et al. use Maximum Mean Discrepancy (MMD) to detect if an API is serving a modified or quantized version of a reference model. If the output distributions deviate significantly, the statistical distance serves as evidence of a "bait-and-switch."
- Rank-Based Uniformity: Xiaoyuan Zhu proposes that if an API serves a model identical to a reference, the rank of the API's tokens within the reference distribution should be uniform. Any deviation from this uniformity allows for the detection of model substitution.
- Privacy Leakage as a Fingerprint: Aoting Hu et al. treat the way a model "remembers" specific training samples—known as privacy leakage—as a unique fingerprint. By applying $p$-value calculations to membership inference queries, they can identify stolen models using only a handful of data samples.
Verifying Dataset Membership
A parallel challenge is proving that a specific dataset was used to train a model. This is particularly difficult with distilled models, which may never see the original data directly. Muxing Li et al. argue that auditors must shift from instance-level memorization to distribution-level statistical alignment, proving that a student model's output distribution is statistically closer to the target dataset than to a control.
For text data, Rastogi et al. show that log-probability analysis on watermarked rephrasings can detect even a single instance of a document in a massive training corpus. Similarly, Bouaziz et al. use a binomial significance test to quantify the probability that a model’s correct responses to secret "taggants" occurred by chance, providing a certificate of usage with extreme statistical certainty.
Why It Works: The Persistence of Bias
These methods work because machine learning models are highly sensitive to the statistical nuances of their training environment. Whether it is the frequency-domain properties of images Liu et al. or the specific feature attributions learned during training Shao et al., these traits are difficult to wash away. By framing ownership as a statistical test rather than a simple comparison, researchers can provide rigorous, legally defensible evidence of provenance in an era of black-box AI.