Mixer: DNN Watermarking using Image Mixup

Kassem Kallas, Teddy Furon

Introduction: The IP Protection Challenge

As Deep Neural Networks (DNNs) become increasingly valuable industrial assets, protecting their Intellectual Property (IP) has become a critical concern. Black-box watermarking is a common defense strategy where a model is trained to respond in a specific, "surprising" way to secret trigger inputs. However, traditional methods often rely on a small, finite set of trigger images. This makes them vulnerable; if an attacker identifies these triggers, they can "unlearn" the watermark through fine-tuning or overwrite it. Furthermore, because these triggers are often out-of-distribution, they are fragile against standard model optimizations like pruning or quantization.

The Mixer Approach: From Points to Manifolds

"Mixer" addresses these vulnerabilities by leveraging Image Mixup, a data augmentation technique that blends multiple images. Instead of using discrete trigger points, Mixer creates an infinite manifold of secret trigger samples.

The watermark embedding process involves blending multiple random images from different classes using a secret weight vector ($\lambda$) and a visible constant overlay ($x_o$). The target label is a "mixed" label ($\tilde{y}$) determined by another secret weight vector ($\mu$). The "secret key" in Mixer is not the images themselves, but the parameters ($\lambda, \mu, x_o$) and the statistical distribution used to generate them.

This approach enables asymmetric verification: the samples used to verify ownership are entirely new and different from those used during training. Because the model learns the underlying logic of the manifold rather than memorizing specific images, it correctly predicts labels for unseen triggers, making it significantly harder for attackers to detect or remove the watermark without destroying the model's primary utility.

Experimental Evaluation

The researchers evaluated Mixer across three benchmarks: MNIST (handwritten digits), CIFAR10 (color images), and a Transfer Learning task where a VGG19 model pre-trained on ImageNet was fine-tuned for CIFAR10.

Key Results and Robustness

Mixer demonstrates high fidelity, maintaining primary task accuracy while achieving near-perfect watermark recovery.

Performance metrics across various model modifications and quantization techniques.

As shown in the results table, the watermark does not degrade primary task performance (e.g., 99.29% on MNIST). In baseline scenarios, the watermark recovery rate ($Rec_{ts}$) reached 100%. Notably, the watermark survived various quantization methods, including Dynamic and Float16 quantization, with minimal loss in detection accuracy.

Robustness to Pruning

The manifold-based approach makes the watermark an integral part of the model's weight distribution. This is evidenced by its resilience to weight pruning.

Accuracy and watermark recovery rates across different pruning rates for MNIST, CIFAR10, and Transfer Learning.

On the MNIST dataset, the model retained over 70% watermark recovery even after 45% of its weights were pruned. This "mutual destruction" property ensures that any attack aggressive enough to remove the watermark—such as Full Int8 quantization—also renders the model's primary task accuracy useless (dropping to random guess levels, around 9%).

Conclusion and Future Work

Mixer represents a shift from discrete trigger points to learned manifolds in DNN watermarking. While highly effective, future work is needed to address "Usurper" attacks where an attacker attempts to claim a fake watermark, and to explore the impact of watermark overwriting. Additionally, theoretical investigations into the learning capacity of networks could help define the upper limits of watermark information density within a manifold.

Summary updated

Abstract

It is crucial to protect the intellectual property rights of DNN models prior to their deployment. The DNN should perform two main tasks: its primary task and watermarking task. This paper proposes a lightweight, reliable, and secure DNN watermarking that attempts to establish strong ties between these two tasks. The samples triggering the watermarking task are generated using image Mixup either from training or testing samples. This means that there is an infinity of triggers not limited to the samples used to embed the watermark in the model at training. The extensive experiments on image classification models for different datasets as well as exposing them to a variety of attacks, show that the proposed watermarking provides protection with an adequate level of security and robustness.
BibTeX
@inproceedings{kallas2023mixer,
  author = {Kassem Kallas and Teddy Furon},
  title = {{Mixer: DNN Watermarking using Image Mixup}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095332},
}
Figure