Performance-lossless Black-box Model Watermarking

Na Zhao, Kejiang Chen, Weiming Zhang, Nenghai Yu

TL;DR

The authors introduce BranchWM, a novel black-box model watermarking protocol that protects the intellectual property of deep learning models without degrading their primary task performance. By decoupling the watermarking mechanism into a parallel API branch triggered by secure Message Authentication Codes (MACs), they achieve a provably performance-lossless verification system that is both computationally efficient and robust against evasion.

What Problem Does This Paper Address?

Training state-of-the-art deep learning models, such as Large Language Models (LLMs), requires massive datasets, specialized expertise, and expensive computational resources. To monetize these assets while protecting their intellectual property (IP), development teams often deploy models behind black-box APIs.

However, dishonest third-party application developers can exploit this setup—for instance, by reselling API access without authorization or using low-quality models to impersonate high-performance ones.

Figure 1: There are three roles in the threat scenario. The model development team provide black-box deployment to the application service provider. The latter may dishonestly provide application services to end users, which we summarize as two cases of dishonesty in the red zone.

To combat this, researchers use model watermarking. Traditional black-box watermarking methods typically inject "backdoors" into the model during training. When the model encounters a specific trigger input, it outputs a predefined watermark signal. The major downside? This entangles the watermark task with the primary task, which inevitably degrades the model's overall accuracy and performance.

This paper addresses a critical question: How can we implement a secure, black-box model watermark that guarantees absolutely zero performance loss on the primary task?

Methods

The authors propose BranchWM, a protocol that shifts the watermarking mechanism away from the model's internal weights and into a parallel branch within the deployment API.

The architecture consists of two main components:

  1. An Outer Module: Used by the model owner to generate trigger queries and verify returned evidence.
  2. An Inner Module: Deployed inside the model API, consisting of a Trigger Detector and an Evidence Generator.
Figure 2: There are two modules in the IP verification protocol. The outer module is deployed outside the model API, responsi- ble for generating triggers to obtain model copyright evidence and verifying the evidence. The inner module is added as a branch in the model API, responsible for checking whether the input is a trigger and generating the copyright evidence.

When a standard user query enters the API, the system remains in its default "Service State," routing the input directly to the original, unaltered model. However, if the Trigger Detector identifies a valid watermark trigger, the API switches to a "Forensic State." The Evidence Generator then injects ownership information into the output.

To make the triggers unforgeable and secure, the authors use a cryptographic primitive: a Message Authentication Code (MAC) that satisfies Existential Unforgeability under Chosen Message Attack (EUF-CMA).

Figure 3: We use the MAC scheme which meets EUF- CMA to construct the Trigger Generator and Trigger Detector sub-modules. Such a construction generates the MUFT that satisfies the DUL-CSA property.

To prevent adversaries from simply filtering out garbled cryptographic strings, the authors integrate a semantic-preserving text generation algorithm (adapted from Kirchenbauer et al.). This ensures that the trigger text looks completely natural and easily bypasses simple perplexity-based anomaly detectors.

Datasets

The language model instantiation of the protocol was evaluated using:

  • ROCStories: A story reasoning dataset used to provide prompt contexts.
  • OPT-1.3B: The pre-trained open transformer language model used to generate the text and embed the concealed triggers.

Key Insights and Ideas

  • Decoupling is Key: By moving the watermark from the model weights to an API-level parallel branch, the primary task performance remains completely untouched. The model behaves exactly as it would without a watermark for 99.99% of normal user queries.
  • Symmetric vs. Asymmetric Crypto: Prior theoretical work relied on digital signatures (asymmetric cryptography) to build undetectable backdoors, which introduces massive computational overhead. This paper demonstrates that symmetric MACs provide the same black-box security guarantees while being orders of magnitude faster.
  • The Cost of Evasion: While an attacker could theoretically use machine-text detectors to filter out potential triggers, doing so would introduce false positives, causing them to drop a large volume of legitimate user queries and severely damaging their own service quality.

Key Results

The authors proved the performance-lossless property mathematically via reduction to the security of the underlying MAC scheme.

Quantitatively, they compared the runtime overhead of different cryptographic primitives to evaluate real-world feasibility. Using SHA512 as a baseline, the verification time rates were measured as follows:

Table I: The runtime cost of the verification time between different cryptographic primitives. We chose one hash function (i.e., SHA512), one MAC scheme (i.e., HMAC SHA512), and two digital signature schemes (i.e., ECDSA and FALCON) according to the recommendations of NIST [52]–[55]. We use SHA512 as the baseline to record the ratios of HMAC, ECDSA and FALCON under the 1024-bit, 163-bit and 10248- bit key lengths, respectively. FALCON 512 is its basic security version.

While digital signature methods like ECDSA introduce a 64.6x slowdown, the proposed HMAC_SHA512 scheme only introduces a negligible 1.2x overhead, making it highly practical for high-throughput APIs.

Limitations and Future Work

The authors note a few limitations to their current approach:

  1. Replay Attacks: Attackers could potentially log previously observed trigger-response pairs and replay the copyright evidence for semantically similar queries. The authors suggest incorporating timestamps or dynamic keys to mitigate this.
  2. Trigger Length Constraints: Due to the capacity limits of current text-watermarking methods, the protocol requires relatively long text inputs to successfully embed the secure cryptographic patterns, making it less suitable for short-form text APIs (like simple chatbots).
  3. Model Extraction Vulnerability: BranchWM assumes the adversary is interacting with the API, but it does not inherently prevent an attacker from training a student model via distillation (model extraction) unless paired with complementary defense frameworks.

Summary updated

Abstract

With the development of deep learning, high-value and high-cost models have become valuable assets, and related intellectual property protection technologies have become a hot topic. However, existing model watermarking work in black-box scenarios mainly originates from training-based backdoor methods, which probably degrade primary task performance. To address this, we propose a branch backdoor-based model watermarking protocol to protect model intellectual property, where a construction based on a message authentication scheme is adopted as the branch indicator after a comparative analysis with secure cryptographic technologies primitives. We prove the lossless performance of the protocol by reduction. In addition, we analyze the potential threats to the protocol and provide a secure and feasible watermarking instance for language models.
BibTeX
@misc{zhao2023performance,
  author = {Na Zhao and Kejiang Chen and Weiming Zhang and Nenghai Yu},
  title = {{Performance-lossless Black-box Model Watermarking}},
  year = {2023},
  eprint = {2312.06488},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.06488},
}
Figure