A Survey on Model Watermarking Neural Networks

Franziska Boenisch

Abstract

Machine learning (ML) models are applied in an increasing variety of domains. The availability of large amounts of data and computational resources encourages the development of ever more complex and valuable models. These models are considered intellectual property of the legitimate parties who have trained them, which makes their protection against stealing, illegitimate redistribution, and unauthorized application an urgent need. Digital watermarking presents a strong mechanism for marking model ownership and, thereby, offers protection against those threats. This work presents a taxonomy identifying and analyzing different classes of watermarking schemes for ML models. It introduces a unified threat model to allow structured reasoning on and comparison of the effectiveness of watermarking methods in different scenarios. Furthermore, it systematizes desired security requirements and attacks against ML model watermarking. Based on that framework, representative literature from the field is surveyed to illustrate the taxonomy. Finally, shortcomings and general limitations of existing approaches are discussed, and an outlook on future research directions is given.

Introduction: The Challenge of Intellectual Property in AI

Training a state-of-the-art neural network is an expensive endeavor, requiring massive datasets, specialized hardware, and significant human expertise. This high value makes trained models lucrative targets for "model stealing" or "model extraction" attacks. In these scenarios, an adversary attempts to replicate a model's functionality to avoid training costs or to resell the service without authorization.

Model extraction attack via surrogate model training.

While traditional digital media has long used watermarking to claim copyright, applying this to the "black box" of a neural network is a relatively new challenge. This paper provides a structured framework for the field of ML watermarking, introducing a common language and a unified threat model to compare different approaches.

A Taxonomy for Model Watermarking

The author systematizes the field by introducing a five-dimensional taxonomy for watermarking schemes:

  1. Embedding Method: Marks can be parameter-based (modifying weights directly) or trigger-based (training the model to exhibit specific "backdoor" behavior on secret inputs).
  2. Verification Access: This determines how an owner checks for a mark. White-box verification requires full access to internal parameters, while black-box verification only requires querying the model via an API.
  3. Capacity: Schemes range from zero-bit (simple presence detection) to multi-bit (carrying complex messages like serial numbers).
  4. Authentication: Whether the mark inherently links to the owner's identity, such as through a digital signature.
  5. Uniqueness: Distinguishing between marks used for all copies versus unique identifiers for individual instances (fingerprinting).
Comparison of white-box (A) and black-box (B) model access scenarios.

Key Insights and Robustness

A core insight of the paper is viewing watermarking as a "good" backdoor—intentionally using adversarial techniques to create a secret proof of ownership. However, this introduces a reliability-integrity trade-off: a watermark must be sensitive enough to detect theft without falsely accusing similar, independently trained models.

Robustness remains a major hurdle. Many early schemes are fragile and can be "laundered" through standard optimizations like fine-tuning, pruning, or quantization. The survey concludes that "entangled" watermarks—where the watermarking task is mathematically woven into the primary task—are the most resistant to such removal attacks.

Conclusion and Future Directions

While the field has matured, current research is heavily biased toward image classification. Future work must address non-image domains like Natural Language Processing (NLP) and Reinforcement Learning. Furthermore, as models scale to billions of parameters, the scalability of these techniques remains an open question. Ultimately, technical watermarks must be integrated with active defenses and legal frameworks to provide comprehensive protection for AI intellectual property.

Summary updated

Abstract

Machine learning (ML) models are applied in an increasing variety of domains. The availability of large amounts of data and computational resources encourages the development of ever more complex and valuable models. These models are considered intellectual property of the legitimate parties who have trained them, which makes their protection against stealing, illegitimate redistribution, and unauthorized application an urgent need. Digital watermarking presents a strong mechanism for marking model ownership and, thereby, offers protection against those threats. This work presents a taxonomy identifying and analyzing different classes of watermarking schemes for ML models. It introduces a unified threat model to allow structured reasoning on and comparison of the effectiveness of watermarking methods in different scenarios. Furthermore, it systematizes desired security requirements and attacks against ML model watermarking. Based on that framework, representative literature from the field is surveyed to illustrate the taxonomy. Finally, shortcomings and general limitations of existing approaches are discussed, and an outlook on future research directions is given.
BibTeX
@article{boenisch2021survey,
  author = {Franziska Boenisch},
  title = {{A Survey on Model Watermarking Neural Networks}},
  journal = {Frontiers Big Data},
  year = {2021},
  doi = {10.3389/fdata.2021.729663},
}
Figure