Introduction
Training state-of-the-art deep neural networks (DNNs) requires a significant investment in data, expertise, and computational resources, making these models valuable intellectual property (IP). However, they are susceptible to theft or illegal redistribution. Current watermarking techniques often fall short: white-box methods require unrealistic full access to model weights, while black-box "trigger set" methods typically offer low information capacity and are easily removed through fine-tuning.
The GradSigns Framework
GradSigns addresses these gaps by embedding digital signatures into the gradient of the cross-entropy cost function with respect to the model's inputs.
Embedding Phase
During training, a "watermark embedding regularizer" is added to the standard loss function. This regularizer aligns the expected gradients of a secret "carrier set" of input neurons with a specific binary signature using a secret transformation matrix (embedding key). This process subtly reshapes the decision boundary without degrading the model's primary task performance.
Verification Phase
Verification can occur in two settings:
- White-box: The owner uses backpropagation to calculate gradients directly.
- Black-box: When only API access is available, the owner employs zeroth-order gradient estimation. By querying the model with an input and its slightly perturbed version, the gradient can be estimated from the difference in prediction scores, effectively revealing internal gradient behavior through external outputs.
Experimental Setup
The framework was evaluated on three image classification tasks:
- CIFAR-10: Object classification using ResNet20.
- SVHN (Street View House Numbers): Digit classification using a 6-layer CNN.
- YTF (YouTube Face): Face recognition using the Deep-ID architecture.
Results and Analysis
GradSigns demonstrates high loyalty, with watermarked models experiencing an average accuracy drop of less than 1% compared to unmarked baselines.
The framework is remarkably robust, maintaining verifiability even after 90% parameter pruning followed by fine-tuning. It also resists adaptive attacks intended to mask gradients by perturbing prediction scores. Statistical analysis confirms high reliability, with a negligible probability (p-value < $3 \times 10^{-3}$) of an unmarked model accidentally containing the signature. Furthermore, the high computational cost for an adversary to forge a watermark into a stolen model preserves the IP's value.
Conclusion
GradSigns leverages the over-parameterization of DNNs to find solutions that carry robust signatures within the gradient signal. While query-intensive in black-box settings, its high capacity (up to 64 bits) and resilience to aggressive tampering make it a powerful tool for IP protection. Future work will explore query efficiency improvements and expansion into other domains like NLP and GANs.
Summary updated