Introduction
As Large Language Models (LLMs) become ubiquitous, "watermarking" their output has emerged as the primary defense for copyright and provenance. This technique subtly shifts the probability distribution (logits) of tokens during generation to embed a statistical "signature" detectable by algorithms but invisible to humans. However, a critical vulnerability exists: watermark collision.
Watermark collision occurs when applying a second watermark—often via downstream tasks like paraphrasing or translation—effectively "erases" the first one. This happens because the underlying probability distributions become tangled, providing a potent new strategy for watermark attacks.
The Problem: A "Wild West" of Watermarks
The authors identify a looming scenario where text is processed by multiple LLMs, each attempting to leave its own mark. This isn't merely an accidental byproduct of multi-model workflows; it is a fundamental weakness that can be exploited as a "collision attack" to strip away existing protections without destroying the text's readability.
Methodology: The Collision Pipeline
The research formalizes a "Collision Pipeline" involving three primary actors:
- The Watermarker ($W$): The original LLM generating watermarked text.
- The Collider ($C$): An attacking LLM that performs a task (e.g., paraphrasing) while simultaneously embedding its own watermark.
- The Detectors: Algorithms designed to identify the specific signatures of $W$ or $C$.
Detection typically relies on a "null hypothesis"—the assumption that natural text follows a random distribution of "green" and "red" tokens. When a second watermark is applied, it creates an entangled distribution. Mathematically, this new distribution no longer conforms to the original watermark’s rules, causing the original detector to fail. The study evaluated three types of colliders: paraphrasers, back-translators, and mask-and-fill models.
Experimental Setup
The study utilized the C4 (Colossal Clean Crawled Corpus) dataset for text generation. Evaluation involved several prominent open-source LLMs, including LLaMA-2-13B, Qwen2-7B, OPT-1.3B, and RoBERTa-LARGE. These were tested against three major watermarking algorithms: KGW (Kirchenbauer et al.), PRW (Zhao et al.), and SIR (Liu et al.).
Key Insights and Results
The results demonstrate that collision is a highly effective and stealthy attack strategy.
- The "Tug-of-War" Effect: Watermarks are competitive rather than additive. When two watermarks overlap, they compete for the "probability budget" of the text, with one usually emerging stronger at the significant expense of the other.
- Detection Collapse: In many scenarios, the True Positive Rate (TPR) of the original watermark dropped from near 100% to below 40% after a single collision with a strong paraphraser.
- Quality Preservation: Unlike traditional "heavy" attacks that scramble text, collision attacks preserve fluency. Perplexity scores remained stable, and in some cases, the double-watermarked text actually showed improved perplexity.
- Multi-round Erasure: Running text through a paraphraser multiple times with different keys acts like a digital sandblaster, progressively eroding the original watermark until it is completely undetectable.
Limitations and Future Outlook
While the study highlights a universal vulnerability across different model architectures, it focused primarily on paraphrasing, translation, and mask-and-fill. Future research is needed to explore how collision behaves in complex tasks like long-form summarization. Currently, there is a "defense gap," as no "collision-proof" watermark yet exists. Developing robust watermarking schemes that can survive or even signal the presence of an overlapping signature remains a critical area for future work.
Summary updated