DeepMarks: A Digital Fingerprinting Framework for Deep Neural Networks

Huili Chen, Bita Darvish Rouhani, Farinaz Koushanfar

Introduction

Training state-of-the-art deep learning models is a resource-intensive process, requiring massive datasets and significant computational power. As these models become valuable intellectual property (IP), protecting them from unauthorized redistribution is critical. While digital watermarking can prove ownership, it fails to perform "traitor tracing"—identifying the specific user responsible for a leak. Furthermore, attackers often employ model pruning, fine-tuning, or collusion attacks (where multiple users combine models to neutralize marks) to bypass security.

DeepMarks is the first end-to-end framework designed to address these challenges. By embedding unique, user-specific binary codes into model weights, it allows owners to trace a redistributed model back to its source, even after significant compression or multi-user collusion.

Methodology

The DeepMarks framework operates through a three-stage pipeline: embedding, identification, and collusion detection.

DeepMarks Global Flow: The three-stage process of fingerprint embedding, user identification, and colluder detection.

1. Fingerprint Generation

Each user is assigned a unique binary code-vector. To defend against collusion, DeepMarks utilizes Balanced Incomplete Block Design (BIBD) to generate Anti-Collusion Codes (ACC). These codes are mathematically structured so that the logical combination of any small subset of codes remains unique to the contributing users.

2. Embedding via Loss Function

Rather than overwriting weights directly, DeepMarks embeds fingerprints into the probability density function of the model weights. This is achieved by adding a regularization term to the standard training loss: $$\mathcal{L} = \mathcal{L}_0 + \gamma MSE(\mathbf{f}_j - \mathbf{Xw})$$ Where $\mathbf{f}_j$ is the fingerprint, $\mathbf{w}$ represents the flattened weights, and $\mathbf{X}$ is a secret random projection matrix. This approach forces the model to learn the primary task while aligning its weight distribution with the assigned fingerprint.

3. Blind Extraction

Identification is a "blind" process; the owner extracts weights from a suspicious model and uses the secret matrix $\mathbf{X}$ to recover the code-vector without needing the original, unmarked model for comparison.

Experimental Evaluation

The framework was evaluated on two standard benchmarks to demonstrate architectural versatility:

  • MNIST: A digit recognition task using a standard Convolutional Neural Network (CNN).
  • CIFAR10: A complex image classification task using a Wide Residual Network (WRN).

Results and Discussion

Fidelity and Performance

Embedding fingerprints did not degrade model accuracy. In fact, the fingerprinting process acted as a form of regularization, slightly improving performance. For MNIST, accuracy rose from 99.52% to 99.72%, and for CIFAR10, it increased from 91.85% to 92.03%.

Collusion Resilience

Using a $(31, 6, 1)$-BIBD code, the system achieved a 100% detection rate and 0% false alarm rate for up to 5 colluders across both datasets. As shown in the performance metrics, the detection capability remains perfect until the theoretical limit of the anti-collusion code is reached.

Detection Rate Performance: The system maintains a 100% detection rate for up to 5 colluders before declining as the theoretical limit of the BIBD code is exceeded.

Robustness to Attacks

The framework proved highly resilient to common removal attempts:

  • Pruning: On the MNIST dataset, the fingerprint remained 100% decodable even after 95% of the weights were pruned.
  • Fine-tuning: Detection rates remained high even when attackers retrained the model to attempt to "wash out" the embedded marks.

Conclusion and Future Work

DeepMarks demonstrates that modern DNNs have significant "extra capacity" that can be exploited for IP protection. By treating fingerprinting as a regularization task, the framework provides robust traitor tracing without sacrificing model utility.

Future research will focus on moving beyond "white-box" requirements (which currently necessitate weight inspection) toward "black-box" fingerprinting, where identification can occur via model API outputs alone. Additionally, scaling BIBD codebooks to support millions of users remains a key area for commercial exploration.

Summary updated

BibTeX
@article{chen2018deepmarks,
  author = {Huili Chen and Bita Darvish Rouhani and Farinaz Koushanfar},
  title = {{DeepMarks: A Digital Fingerprinting Framework for Deep Neural Networks}},
  journal = {IACR Cryptol. ePrint Arch.},
  year = {2018},
  url = {https://openreview.net/forum?id=_ZHPe_nTHM2},
}
Figure