Introduction
Contrastive learning methods, such as SimCLR and MoCoV2, have significantly advanced computer vision by enabling the training of powerful feature encoders on vast quantities of unlabeled data. However, the high computational cost and data requirements make these pre-trained encoders valuable intellectual property (IP). Protecting this IP is challenging because traditional watermarking techniques, which typically rely on supervised "backdoor attacks" to force specific label outputs, are ill-suited for task-agnostic encoders that produce feature vectors rather than class labels.
The Proposed Framework
The authors introduce a novel methodology to embed "untargeted" backdoors into the encoder's feature space. This approach ensures that any downstream model derived from a stolen encoder will exhibit predictable, verifiable errors when presented with a specific trigger, regardless of the final application.
The watermarking process involves fine-tuning a pre-trained encoder $f$ into a watermarked version $f'$ using a dual-objective loss function:
- Uniqueness Loss ($L_u$): Forces the encoder to generate a feature vector for a triggered image that is maximally distinct from the feature vector of its clean counterpart.
- Functionality-Preserving Loss ($L_p$): Ensures that for clean images, the watermarked encoder's output remains nearly identical to the original, preserving the model's utility for downstream tasks.
To enhance robustness, Dropout is incorporated during the embedding phase to simulate potential modifications like model pruning or further fine-tuning. Verification is performed by measuring Watermark Accuracy (WACC): if a suspicious model consistently predicts different labels for triggered vs. clean versions of the same sample, ownership is confirmed.
Experimental Results
The methodology was evaluated using ResNet18 (SimCLR) and ResNet50 (MoCoV2) architectures across datasets including CIFAR-10, ImageNet, STL10, GTSRB, and SVHN.
Key findings from the experiments include:
- Effectiveness: Watermarked models achieved high WACC (typically 80-94%), whereas independent "clean" models showed significantly lower WACC (often below 50%), facilitating clear IP verification.
- Utility Preservation: The impact on primary task accuracy (ACC) was negligible, with differences usually within ±1%. Interestingly, the authors found that triggered samples sometimes acted as a form of regularization, slightly improving downstream performance.
- Robustness: The watermark remained verifiable even after aggressive model pruning (up to 80% of parameters removed) and survived common adaptation techniques like "Fine-Tune All Layers" (FTAL) and "Re-Train Last Layers" (RTLL).
Conclusion and Future Work
By shifting the focus from supervised label manipulation to latent space manipulation, this work provides a robust solution for protecting task-agnostic encoders in the era of self-supervised learning. Future research will focus on optimizing trigger patterns to further minimize false positives and developing defenses against increasingly sophisticated adaptive attacks designed to "wash out" embedded watermarks.
Summary updated