Introduction
Training state-of-the-art deep learning models is a resource-intensive process, requiring massive datasets and significant computational power. As these models become valuable intellectual property (IP), protecting them from unauthorized redistribution is critical. While digital watermarking can prove ownership, it fails to perform "traitor tracing"—identifying the specific user responsible for a leak. Furthermore, attackers often employ model pruning, fine-tuning, or collusion attacks (where multiple users combine models to neutralize marks) to bypass security.
DeepMarks is the first end-to-end framework designed to address these challenges. By embedding unique, user-specific binary codes into model weights, it allows owners to trace a redistributed model back to its source, even after significant compression or multi-user collusion.
Methodology
The DeepMarks framework operates through a three-stage pipeline: embedding, identification, and collusion detection.
1. Fingerprint Generation
Each user is assigned a unique binary code-vector. To defend against collusion, DeepMarks utilizes Balanced Incomplete Block Design (BIBD) to generate Anti-Collusion Codes (ACC). These codes are mathematically structured so that the logical combination of any small subset of codes remains unique to the contributing users.
2. Embedding via Loss Function
Rather than overwriting weights directly, DeepMarks embeds fingerprints into the probability density function of the model weights. This is achieved by adding a regularization term to the standard training loss: $$\mathcal{L} = \mathcal{L}_0 + \gamma MSE(\mathbf{f}_j - \mathbf{Xw})$$ Where $\mathbf{f}_j$ is the fingerprint, $\mathbf{w}$ represents the flattened weights, and $\mathbf{X}$ is a secret random projection matrix. This approach forces the model to learn the primary task while aligning its weight distribution with the assigned fingerprint.
3. Blind Extraction
Identification is a "blind" process; the owner extracts weights from a suspicious model and uses the secret matrix $\mathbf{X}$ to recover the code-vector without needing the original, unmarked model for comparison.
Experimental Evaluation
The framework was evaluated on two standard benchmarks to demonstrate architectural versatility:
- MNIST: A digit recognition task using a standard Convolutional Neural Network (CNN).
- CIFAR10: A complex image classification task using a Wide Residual Network (WRN).
Results and Discussion
Fidelity and Performance
Embedding fingerprints did not degrade model accuracy. In fact, the fingerprinting process acted as a form of regularization, slightly improving performance. For MNIST, accuracy rose from 99.52% to 99.72%, and for CIFAR10, it increased from 91.85% to 92.03%.
Collusion Resilience
Using a $(31, 6, 1)$-BIBD code, the system achieved a 100% detection rate and 0% false alarm rate for up to 5 colluders across both datasets. As shown in the performance metrics, the detection capability remains perfect until the theoretical limit of the anti-collusion code is reached.
Robustness to Attacks
The framework proved highly resilient to common removal attempts:
- Pruning: On the MNIST dataset, the fingerprint remained 100% decodable even after 95% of the weights were pruned.
- Fine-tuning: Detection rates remained high even when attackers retrained the model to attempt to "wash out" the embedded marks.
Conclusion and Future Work
DeepMarks demonstrates that modern DNNs have significant "extra capacity" that can be exploited for IP protection. By treating fingerprinting as a regularization task, the framework provides robust traitor tracing without sacrificing model utility.
Future research will focus on moving beyond "white-box" requirements (which currently necessitate weight inspection) toward "black-box" fingerprinting, where identification can occur via model API outputs alone. Additionally, scaling BIBD codebooks to support millions of users remains a key area for commercial exploration.
Summary updated