Abstract
The rapid advancement of deep learning relies heavily on high-quality, large-scale datasets. However, protecting the intellectual property (IP) of these datasets remains a significant challenge, as traditional watermarking techniques often fail when only the final trained model is accessible. This paper introduces DVBW (Dataset Verification via Backdoor Watermarking), a novel framework for black-box ownership verification. By embedding "poison-only" backdoor triggers into a small fraction of the training data, DVBW enables data owners to prove a model was trained on their dataset through statistical hypothesis testing of model predictions. Our evaluation across image, text, and graph modalities shows that DVBW achieves high verification accuracy with negligible impact on primary task performance and remains robust against common model-level defenses.
Introduction
Data is the cornerstone of modern artificial intelligence. While datasets are frequently released under restrictive licenses, enforcing these terms is difficult once the data has been used to train a proprietary model. Existing protection methods, such as encryption or differential privacy, often degrade data utility or are inapplicable when the "thief" only provides black-box access to the resulting model. The authors address this by repurposing backdoor attacks—typically viewed as a security threat—into a proactive defense mechanism for dataset IP protection.
Methodology: The DVBW Framework
The DVBW pipeline consists of two primary phases: dataset watermarking and ownership verification.
1. Dataset Watermarking
In the first phase, a defender selects a small subset of the benign dataset ($D_s$) and applies a "poison-only" backdoor attack. This involves adding a specific trigger (e.g., a pixel pattern or a keyword) to the samples and changing their labels to a pre-defined "target label." When a third party trains a model on this released victim dataset, the model inadvertently learns to associate the trigger with the target label, effectively embedding a hidden watermark.
2. Dataset Verification
To verify a suspicious model, the defender queries it with watermarked samples and analyzes the outputs:
- Probability-Available Verification: When confidence scores are accessible, a pair-wise T-test compares the model's confidence in the target label for triggered vs. benign samples.
- Label-Only Verification: In more restrictive settings where only class labels are provided, a Wilcoxon-test is employed to detect the presence of the backdoor statistically.
Experimental Evaluation
The authors evaluated DVBW across diverse data modalities to ensure its generalizability, including image recognition (CIFAR-10, ImageNet), Natural Language Processing (IMDB, DBpedia), and graph recognition (COLLAB, REDDIT).
As shown in Figure 3, the triggers used in image datasets (like BadNets or Blended patterns) are designed to be subtle, ensuring the dataset's utility for legitimate users remains high while remaining detectable by the owner.
Key Results and Insights
- High Verification Success: The Watermark Success Rate (WSR) consistently exceeded 94%, reaching over 99% in many configurations. This indicates that models trained on watermarked data reliably exhibit the backdoor behavior.
- Minimal Utility Loss: The impact on Benign Accuracy (BA) was minimal, typically dropping by less than 1%. This confirms that DVBW does not significantly hinder the primary training task.
- Robustness: The embedded watermarks proved resilient to common post-training modifications, including model pruning and fine-tuning of the final layers.
- Statistical Significance: In "steal" scenarios, the p-values obtained during verification were extremely low (e.g., $10^{-100}$), providing overwhelming evidence of unauthorized data use.
Conclusion and Future Work
DVBW successfully transforms backdoor attacks into a robust tool for dataset IP protection. By working in a black-box setting across multiple modalities, it offers a practical solution for data owners to monitor and enforce licensing agreements. Future research will focus on increasing the stealthiness of watermarks against adaptive defenses like Anti-Backdoor Learning (ABL) and optimizing the poisoning rate to balance verification strength with model utility.
Summary updated