Introduction
As Large Language Models (LLMs) become integrated into daily information consumption, the ability to distinguish machine-generated text from human writing is essential for maintaining information integrity and protecting intellectual property. Traditional watermarking techniques embed signals by subtly biasing token selection. However, these methods typically rely on a "shared secret" key. If this key is made public for verification purposes, it can be exploited by malicious actors to forge the watermark, falsely attributing synthetic content to a trusted source.
This paper introduces UPV (Unforgeable Publicly Verifiable), a novel watermarking system that resolves this tension. By employing a pair of distinct neural networks—a private one for embedding and a public one for detection—UPV enables universal verification without compromising the security of the generation process.
Methodology: The UPV Framework
The core innovation of UPV lies in its computational asymmetry, drawing inspiration from one-way functions in cryptography. The system consists of two primary components:
- The Private Generator: A small Feed-Forward Network (FFN) that analyzes a local window of preceding tokens to determine a "green list" for the next token. The LLM's logits are then biased to favor these tokens.
- The Public Detector: An LSTM-based network trained to identify the watermark patterns. Crucially, while the detector can recognize the watermark, reverse-engineering the generator's specific rules from the detector is computationally prohibitive.
To enhance efficiency, both networks share the same token embedding parameters. This allows the detector to achieve high accuracy with a minimal parameter footprint (~43k parameters). Additionally, the system utilizes "cyclic labeling" to prevent attackers from probing the detector to deduce generation rules.
Experimental Setup
The researchers evaluated UPV using models of varying scales, including GPT-2, OPT-1.3B, and LLaMA-7B. Testing was conducted on the C4 (Colossal Clean Crawled Corpus) for general performance and the DBPEDIA CLASS dataset to assess domain robustness. Performance was benchmarked against standard key-based watermarking methods.
Results and Discussion
UPV demonstrates exceptional performance across several critical metrics:
- Detection Accuracy: The system achieves a near-perfect F1 score of 99%, matching the effectiveness of traditional key-based methods.
- Unforgeability: "Reverse training" attacks, where an adversary attempts to train a surrogate generator using the public detector, consistently failed to produce high-quality forgeries.
- Text Quality: The impact on text naturalness is negligible. Metrics such as perplexity (PPL) and BLEU scores show only marginal changes, ensuring the watermark does not degrade the LLM's utility.
- Robustness: The watermark remains detectable even after text is rewritten by other models like GPT-3.5, often outperforming existing techniques in these scenarios.
Limitations
While highly effective, the system's security is sensitive to the token window size ($w$). Smaller windows may be susceptible to frequency analysis. Furthermore, while robust against single rewrites, the system's resilience against iterative adversarial attacks or sophisticated "copy-paste" strategies requires further investigation.
Conclusion
UPV represents a significant step forward in LLM transparency. By decoupling the generation and detection mechanisms, it provides a practical path toward public verification of AI-generated content. This "unforgeable" approach ensures that while anyone can verify the origin of a text, the power to create a valid watermark remains securely in the hands of the model provider.
Summary updated