Towards the Vulnerability of Watermarking Artificial Intelligence Generated Content

Guanlin Li, Yifei Chen, Jie Zhang, Jiwei Li, Shangwei Guo, Tianwei Zhang

Abstract

AI-Generated Content (AIGC) is gaining great popularity, with many emerging commercial services and applications. These services leverage advanced generative models, such as latent diffusion models and large language models, to generate creative content for users. The usage of such generated content needs to be highly regulated, as the service providers need to ensure the users do not violate the usage policies. A promising solution to achieve this goal is watermarking, which adds unique and imperceptible watermarks on the content for service verification and attribution. Numerous watermarking approaches have been proposed recently. However, in this paper, we show that an adversary can easily break these watermarking mechanisms. Specifically, we consider two possible attacks. (1) Watermark removal: the adversary can easily erase the embedded watermark from the generated content and then use it freely bypassing the regulation of the service provider. (2) Watermark forging: the adversary can create illegal content with forged watermarks from another user, causing the service provider to make wrong attributions. We propose Warfare, a unified methodology to achieve both attacks in a holistic way. The key idea is to leverage a pre-trained diffusion model for content processing and a generative adversarial network for watermark removal or forging. We evaluate Warfare on different datasets and embedding setups. The results prove that it can achieve high success rates while maintaining the quality of the generated content. Compared to existing diffusion model-based attacks, Warfare is 5,050~11,000x faster.

Introduction

As AI-Generated Content (AIGC) explodes, so do concerns about intellectual property theft, fake news, and malicious deepfakes. Watermarking has emerged as the primary "shield"—a way for service providers to bake invisible tracking data into images to verify their origin. However, current watermarking techniques may be more fragile than they appear. Previous attacks on watermarks were often slow, required knowledge of the specific watermarking algorithm, or needed a "clean" (unwatermarked) version of the data to train on—conditions rarely met in real-world scenarios. This research introduces a practical, "black-box" threat model where an attacker can break these protections using only the watermarked images available online.

Methodology: The Warfare Framework

The researchers propose Warfare, a unified attack framework designed to both erase invisible watermarks and "forge" them to frame innocent users. The core innovation is a three-step pipeline that operates without access to original unwatermarked data.

Overview of the Warfare framework, illustrating data collection, pre-processing via diffusion models, and GAN-based model training for removal and forging.
  1. Data Collection: The attacker harvests watermarked images ($x'$) from a target service or user.
  2. Data Pre-processing (The "Mediator"): Since clean images are unavailable, a pre-trained diffusion model (e.g., DDPM) is used to heavily denoise the watermarked images. By adding massive noise and then denoising, the watermark signal is effectively destroyed, creating a "mediator" dataset ($\hat{x}$). These images serve as a proxy for clean data.
  3. Model Training: A Generative Adversarial Network (GAN) is trained using the collected and mediator data.
  4. Watermark Removal: The GAN learns to map watermarked images back to the "clean" mediator distribution.
  5. Watermark Forging: The GAN learns the reverse: how to apply the specific watermark signature of a victim onto arbitrary images.

An optimized version, Warfare-Plus, improves efficiency by replacing the slow diffusion denoising step with unconditional sampling from Stable Diffusion 1.5. This generates mediator images representing the general distribution of natural images, significantly reducing pre-processing time.

Experimental Setup

The framework was evaluated across several standard and high-resolution datasets:

  • CIFAR-10 & CelebA: Baseline testing on low-resolution images.
  • LSUN-bedroom: Performance testing on complex, 256x256 real-world scenes.
  • AIGC Datasets: Images generated by Stable Diffusion 1.5 and 2.1 to test against industry-standard models.

Key Results and Discussion

Warfare demonstrates high attack success rates while preserving image quality. Against StegaStamp, it reduced watermark detection accuracy (Bit Acc) from 100% to approximately 47-50% for removal—effectively random guessing. For forging, it achieved over 90% accuracy in successfully embedding a victim's mark.

Comparison of Warfare against various baseline attack methods across different metrics for watermark removal and forging.

As shown in the comparative results, Warfare significantly outperforms traditional image processing attacks (like JPEG compression or Gaussian noise) and existing diffusion-based methods in terms of the trade-off between watermark destruction and image quality preservation.

Furthermore, Warfare-Plus achieved a massive speedup, reducing total attack time by 80% to 85%. For a dataset of 10,000 images at 512x512 resolution, pre-processing time dropped from over 42 hours to just 1.68 hours.

Conclusion

The Warfare framework highlights a critical vulnerability in current AIGC watermarking: the "Mediator" strategy allows attackers to bypass the need for clean training data. The ability to forge watermarks presents a particularly severe threat, as it enables malicious actors to frame innocent users for generating illicit content. While high-resolution training remains a computational challenge, this work serves as a necessary "red-teaming" exercise, urging the development of more resilient, attack-aware watermarking technologies.

Summary updated

Abstract

AI-Generated Content (AIGC) is gaining great popularity, with many emerging commercial services and applications. These services leverage advanced generative models, such as latent diffusion models and large language models, to generate creative content (e.g., realistic images and fluent sentences) for users. The usage of such generated content needs to be highly regulated, as the service providers need to ensure the users do not violate the usage policies (e.g., abuse for commercialization, generating and distributing unsafe content). A promising solution to achieve this goal is watermarking, which adds unique and imperceptible watermarks on the content for service verification and attribution. Numerous watermarking approaches have been proposed recently. However, in this paper, we show that an adversary can easily break these watermarking mechanisms. Specifically, we consider two possible attacks. (1) Watermark removal: the adversary can easily erase the embedded watermark from the generated content and then use it freely bypassing the regulation of the service provider. (2) Watermark forging: the adversary can create illegal content with forged watermarks from another user, causing the service provider to make wrong attributions. We propose Warfare, a unified methodology to achieve both attacks in a holistic way. The key idea is to leverage a pre-trained diffusion model for content processing and a generative adversarial network for watermark removal or forging. We evaluate Warfare on different datasets and embedding setups. The results prove that it can achieve high success rates while maintaining the quality of the generated content. Compared to existing diffusion model-based attacks, Warfare is 5,050~11,000x faster.
BibTeX
@misc{li2023towards,
  author = {Guanlin Li and Yifei Chen and Jie Zhang and Jiwei Li and Shangwei Guo and Tianwei Zhang},
  title = {{Towards the Vulnerability of Watermarking Artificial Intelligence Generated Content}},
  year = {2023},
  eprint = {2310.07726},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.07726},
}
Figure