Introduction
Training state-of-the-art deep neural networks (DNNs) requires significant computational resources and proprietary datasets, making model theft a major intellectual property concern. To combat this, researchers use "fingerprinting"—generating specific inputs that trigger predictable responses from a model to verify ownership. However, existing methods often fail due to a "Goldilocks" problem: they are either too fragile to survive common model modifications like pruning, or they transfer too easily to unrelated models, leading to false accusations of theft.
Methodology
To address these challenges, this research introduces Characteristic Examples (C-examples). Unlike traditional watermarking, C-examples are data-free and generated after a model is trained. The generation process evolves through three key stages to optimize for robustness and uniqueness:
- Vanilla C-examples: These are generated from random noise using Projected Gradient Descent (PGD) to find inputs that the target model classifies with high confidence into a randomly assigned label.
- Robust C-examples (RC-examples): To ensure the fingerprint survives model compression, the authors inject random noise into the model's weights during generation. This forces the fingerprint to rely on stable features that persist even under parameter perturbation.
- Low-Transferability RC-examples (LTRC-examples): To prevent false positives, a high-pass filter is applied in the frequency domain via Discrete Cosine Transform (DCT). By masking low-frequency components—which are typically shared across different models—the fingerprint becomes uniquely tied to the specific architecture and weights of the target model.
The researchers evaluated these methods on ImageNet (VGG-16) and CIFAR-10 (CNN-1) benchmarks, testing transferability against variant models like VGG-19, ResNet, and DenseNet.
Key Insights and Results
The core innovation lies in the Frequency Trade-off. Standard adversarial attacks often rely on low-frequency features that are universal across models. By forcing the fingerprint into the high-frequency spectrum, it becomes a fine-grained signature of one specific model instance. Additionally, treating pruning as a form of weight noise during generation allows the fingerprint to be "pre-hardened" against future compression.
Key performance metrics include:
- Superior Uniqueness: LTRC-examples achieved a "Uniqueness Score" (Robustness minus Transferability) four times higher than baseline methods on VGG ImageNet models.
- Resilience to Pruning: Even with 80% model pruning, LTRC fingerprints maintained high accuracy, while traditional sensitive samples dropped to near-zero robustness (0.04%).
- Zero False Alarms: The LTRC method achieved an AUC and F1-score of 1.0 against unrelated models, perfectly distinguishing the owner's pruned models from different architectures.
Conclusion
Characteristic Examples provide a robust, non-invasive, and data-free solution for DNN intellectual property protection. While the current work focuses on pruning, future research may explore resilience against fine-tuning or knowledge distillation, as well as extending these high-frequency signatures to non-image domains like Natural Language Processing.
Summary updated