Spoofing and Forgery of Generative AI Watermarks

40 concepts23 papers1 proposal

As generative artificial intelligence becomes deeply integrated into content creation, distinguishing between human and machine-generated media has become a critical challenge. One of the most promising solutions is digital watermarking—the practice of embedding imperceptible, mathematically verifiable signatures directly into the outputs of large language models (LLMs) and diffusion models.

However, a major security flaw threatens this paradigm. If a detection system can recognize a watermark, an adversary can often reverse-engineer its underlying rules and replicate it. This vulnerability enables watermark spoofing (or signature forgery), where an attacker intentionally injects a model's signature into human-authored text or authentic images. Instead of merely erasing a watermark to evade detection, spoofing allows malicious actors to weaponize provenance tools, triggering false positives to frame innocent users or discredit model providers.

How Watermarks are Stolen

To forge a watermark, an adversary must first extract its generation rules. Most text watermarking schemes operate by partitioning a model's vocabulary into "green-lists" and "red-lists" based on the context of preceding tokens (n-grams), subsequently biasing token selection toward the green-list. Because this biasing is deterministic or follows a predictable pseudo-random pattern, it leaves a statistical footprint.

An attacker can treat a watermarked model as a black box. By submitting repeated queries and analyzing how the output token distributions shift compared to an unwatermarked reference model, the attacker can systematically reconstruct the scoring function used to bias the tokens. As demonstrated by Nikola Jovanovic 0001 et al. in Watermark Stealing in Large Language Models, an adversary can fully reverse-engineer secret watermarking rules through an API for less than fifty dollars, gaining the ability to forge the signature at will.

A similar vulnerability exists in the visual domain. Semantic watermarks for diffusion models embed signatures into the latent spaces or frequency domains of images. Andreas Müller 0025 et al. show in Black-Box Forgery Attacks on Semantic Watermarks for Diffusion Models that an attacker can extract a watermark from a single reference image and successfully imprint it onto entirely unrelated content generated by a completely different model.

The Mechanics of Forgery

Once the signature's underlying pattern is exposed, the process of forgery is straightforward: the adversary injects these statistical preferences into clean, non-synthetic data.

[Target Model API] ──(Queries)──> [Attacker] ──(Statistical Inference)──> [Extracted Rules]
                                                                                │
                                                                                ▼
[Human Text / Real Image] ─────────────(Inject Rules/Noise)────────────────> [Spoofed Content]
                                                                                │
                                                                                ▼
                                                                     [Detector: FALSE POSITIVE]

In text generation, frameworks like DE-MARK, developed by Ruibo Chen et al. in De-mark: Watermark Removal in Large Language Models, leverage extracted parameters to force unwatermarked text to conform to a target model's token preferences. For images, Mehrdad Saberi et al. demonstrate in Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks that attackers can generate noise patterns representing a specific digital watermark and blend them into authentic, real-world photographs, tricking detectors into flagging them as AI-generated.

This capability introduces severe reputational and legal risks:

  • Framing Model Providers: As explored by Chenchen Gu et al. in On the Learnability of Watermarks for Language Models, an adversary can distill a target model's signature into a separate, malicious model. This model can then be used to generate hate speech or propaganda that carries the reputable provider's signature, falsely attributing the harmful content to them.
  • Targeting Individuals: Vinu Sankar Sadasivan et al. show in Can AI-Generated Text be Reliably Detected? that injecting these patterns into human-authored text triggers false positives in plagiarism detectors, potentially ruining the reputations of students or writers.
  • Cross-Domain Misdirection: The risk of signature mimicry extends beyond multimedia. In biosecurity, Gregory W. Lewis et al. note in The biosecurity benefits of genetic engineering attribution that sophisticated actors could intentionally mimic the unique methodological design habits of rival laboratories in engineered DNA sequences to misdirect forensic blame during an investigation.

Designing Unforgeable Signatures

To mitigate the threat of spoofing, researchers are shifting away from symmetric watermarks—where the information used to detect a signature is identical to the information used to generate it—toward more secure architectures.

Asymmetric and Publicly Verifiable Watermarks

If a detection key is public, an adversary can typically exploit it to reverse-engineer the generation process. To break this dependency, Aiwei Liu et al. introduced a publicly verifiable watermarking scheme (An Unforgeable Publicly Verifiable Watermark for Large Language Models) that utilizes two distinct neural networks. One network privately generates the watermark, while a separate, asymmetric network handles public verification. Because the generation rules remain hidden behind a secret key, exposing the verification mechanism does not grant attackers the ability to forge the signature.

Statistical Stealth

An alternative defense involves making watermarks statistically invisible, preventing attackers from mapping out token preferences. Traditional watermarks alter text distributions globally, leaving obvious statistical clues. To address this, Ya Jiang et al. developed StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models, which embeds metadata without altering the text's underlying statistical distribution.

Similarly, Huanming Shen proposed the SEEK framework in Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks. By using sub-vocabulary decomposition and equivalent texture keys, the author eliminates predictable word-boosting patterns, making it mathematically difficult for an adversary to isolate and copy the watermark rules.

Detecting the Forgeries

Defenders can also exploit the inherent imperfections of spoofing attacks. Because an attacker rarely possesses perfect knowledge of a model's exact parameters or secret keys, their forged signatures are usually approximations. Thibaud Gloaguen et al. demonstrate in Discovering Spoofing Attempts on Language Model Watermarks that learning-based spoofing attempts leave behind distinct statistical "artifacts." By training detectors to identify these subtle anomalies, defenders can reliably distinguish between genuine AI-generated content and malicious fabrications.

Key Foundational Work

Can AI-Generated Text be Reliably Detected? (2023) — contributed 4 concept elements to this direction.
Towards the Vulnerability of Watermarking Artificial Intelligence Generated Content (2023) — contributed 3 concept elements to this direction.
The biosecurity benefits of genetic engineering attribution (2020) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_watermark_spoofing_f4cdf6eb,
  title = {watermark spoofing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/watermark-spoofing/rdr_f4cdf6ebdbc34a66833a051c4c3f4c07},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_watermark_spoofing_f4cdf6eb_v1,
  title = {watermark spoofing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/watermark-spoofing/rdr_f4cdf6ebdbc34a66833a051c4c3f4c07/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake