Abstract
Diffusion Models (DMs) have shown remarkable capabilities in various image-generation tasks. However, there are growing concerns that DMs could be used to imitate unauthorized creations and thus raise copyright issues. To address this issue, we propose a novel framework that embeds personal watermarks in the generation of adversarial examples. Such examples can force DMs to generate images with visible watermarks and prevent DMs from imitating unauthorized images. We construct a generator based on conditional adversarial networks and design three losses (adversarial loss, GAN loss, and perturbation loss) to generate adversarial examples that have subtle perturbation but can effectively attack DMs to prevent copyright violations. Training a generator for a personal watermark by our method only requires 5-10 samples within 2-3 minutes, and once the generator is trained, it can generate adversarial examples with that watermark significantly fast (0.2s per image). We conduct extensive experiments in various conditional image-generation scenarios. Compared to existing methods that generate images with chaotic textures, our method adds visible watermarks on the generated images, which is a more straightforward way to indicate copyright violations. We also observe that our adversarial examples exhibit good transferability across unknown generative models. Therefore, this work provides a simple yet powerful way to protect copyright from DM-based imitation.
Introduction
The rapid advancement of generative Diffusion Models (DMs) has enabled the high-fidelity recreation of artistic styles and specific works, often without the consent of the original creators. Techniques such as "Textual Inversion" and "DreamBooth" pose a direct threat to intellectual property. While existing protections like invisible watermarking exist, they fail to deter imitation proactively. Conversely, current adversarial attacks like AdvDM or Mist produce "chaotic textures" that are difficult for users to interpret as copyright warnings. This paper introduces a novel framework that bridges this gap by embedding visible, interpretable watermarks into the outputs of DMs through subtle adversarial perturbations on the source images.
Methodology
The core of our approach is a conditional GAN-based generator ($G$) designed for efficiency and interpretability. Unlike previous optimization-based methods that require significant computation for each image, our generator is trained once for a specific watermark and can then process images in real-time.
The training process utilizes three distinct loss functions:
- Adversarial Loss ($\mathcal{L}_{adv}$): Tricked the DM into perceiving the watermark in its latent space, forcing it to manifest in the generated output.
- GAN Loss ($\mathcal{L}_{GAN}$): Maintains the visual quality of the protected image, ensuring the perturbation remains stealthy to human observers.
- Weighted Perturbation Loss ($\mathcal{L}_{pert}$): Constrains the magnitude of the added noise, with higher weights in watermark regions to preserve image fidelity.
Results and Discussion
We evaluated our framework on the WikiArt and ImageNet datasets, comparing it against state-of-the-art methods like AdvDM and Mist.
Performance and Visibility
Our method achieved significantly higher watermark visibility in DM outputs, with Normalized Cross-Correlation (NCC) scores of 0.31–0.40, compared to Mist's 0.09–0.15. Qualitatively, the generated images clearly display the intended copyright text (e.g., "REMBRANDT"), providing an unambiguous warning of infringement.
Efficiency and Transferability
The generator-based architecture allows for extremely fast protection, clocking in at 0.2 seconds per image—roughly 150x faster than iterative optimization methods. Furthermore, the adversarial examples demonstrated strong transferability, remaining effective against various model architectures like NovelAI and Dreamshaper, even when trained on Stable Diffusion 1.5.
Conclusion
This research presents a powerful and practical tool for artists to safeguard their creative works against unauthorized AI imitation. By forcing DMs to produce visible watermarks, the method provides a clear, traceable, and interpretable deterrent. Future work will focus on enhancing robustness in black-box scenarios and further optimizing the trade-off between perturbation stealthiness and watermark visibility.
Summary updated