As Large Language Models (LLMs) are increasingly used to generate everything from news articles to code, the need to distinguish machine-generated text from human-written content has become a matter of safety and accountability. The primary technical solution is watermarking: subtly biasing the model's token selection so that the resulting text contains a statistical "signature" invisible to humans but detectable by a mathematical test.
However, recent research reveals a fundamental security flaw in these systems. The very signals that allow a provider to claim "this is mine" can be reverse-engineered by adversaries to either erase the signature (scrubbing) or, more dangerously, forge it onto harmful content (spoofing).
The Vulnerability of Statistical Rules
Most current watermarking schemes rely on a "secret" rule—often based on n-grams—that partitions the model's vocabulary into "green" and "red" lists. During generation, the model is encouraged to pick tokens from the green list. Detection then becomes a simple statistical test: if a text has an improbable number of green tokens, it is flagged as AI-generated.
The problem, as identified by Nikola Jovanovic 0001 and Martin T. Vechev in Watermark Stealing in Large Language Models, is that these rules are not as secret as they seem. By querying a watermarked API and observing which words are boosted in specific contexts, an attacker can "steal" the watermark's logic for less than $50. Once the green-red list logic is exposed, the watermark is no longer a secure lock; it is a public blueprint.
Spoofing: The Reputation Risk
The most critical threat identified across this research is watermark spoofing. If an adversary can reverse-engineer the watermarking rules, they can apply those same statistical biases to human-written text or outputs from a different, perhaps malicious, model.
Chenchen Gu and Tatsunori Hashimoto demonstrate in On the Learnability of Watermarks for Language Models that an attacker can even "distill" a watermark into a surrogate model's weights. This allows the attacker to generate high volumes of toxic or prohibited content that carries the statistical signature of a reputable provider. This creates a "framing" risk: a model developer could be held legally or reputationally responsible for harmful content they never actually generated.
The Robustness Paradox
In an attempt to make watermarks more useful, researchers often aim for robustness—the ability of a watermark to survive edits, paraphrasing, or translation. However, Tong Zhou and Shaolei Ren point out a "robustness paradox" in Bileve.
If a watermark is designed to be highly resilient to changes, it becomes a "piggyback" vector. An attacker can take a legitimate, watermarked output from a model and append or inject malicious content into it. Because the detector is designed to be tolerant of modifications, the final hybrid text—now containing harmful misinformation—will still trigger a positive detection, falsely attributing the entire message to the original model provider.
Scrubbing and the Limits of Detection
While spoofing targets the provider's reputation, scrubbing targets the detector's efficacy. Frameworks like DE-MARK by Ruibo Chen and B4 by Baizhou Huang show that once the watermark rules are inferred, they can be systematically neutralized. By replacing "green" tokens with "red" ones of similar meaning, attackers can strip the watermark while maintaining the text's quality.
This leads to a sobering theoretical conclusion. Vinu Sankar Sadasivan and Aounon Kumar argue in Can AI-Generated Text be Reliably Detected? that as LLMs become better at mimicking the human distribution of language, the gap between "AI-like" and "human-like" text vanishes. In this regime, any watermark that is "invisible" (i.e., doesn't degrade text quality) becomes fundamentally vulnerable to being spoofed by humans or scrubbed by other AI.
Emerging Defenses: Beyond Simple Statistics
To counter these vulnerabilities, the field is moving toward more complex, multi-layered signatures:
- Publicly Verifiable Watermarks: Aiwei Liu and colleagues proposed UPV, which uses asymmetric neural networks to allow anyone to verify a watermark without giving them the power to forge it.
- Bi-level Signatures: The Bileve framework combines a robust statistical signal for general tracing with a fragile cryptographic signature for tamper detection, allowing providers to prove if a text was modified after generation.
- Artifact Discovery: Thibaud Gloaguen and Robin Staab suggest in Discovering Spoofing Attempts that spoofed text leaves behind distinct statistical "artifacts" because attackers lack the true private key, providing a post-hoc way to distinguish forgeries from genuine outputs.
The research consensus suggests that watermarking is not a "set and forget" solution. Instead, it is an evolving arms race where the security of a digital signature depends as much on its resistance to imitation as it does on its resistance to removal.