Abstract
Large Language Models (LLMs) are widely used in complex natural language processing tasks but raise privacy and security concerns due to the lack of identity recognition. This paper proposes a multi-party credible watermarking framework (CredID) involving a trusted third party (TTP) and multiple LLM vendors to address these issues. In the watermark embedding stage, vendors request a seed from the TTP to generate watermarked text without sending the user's prompt. In the extraction stage, the TTP coordinates each vendor to extract and verify the watermark from the text. This provides a credible watermarking scheme while preserving vendor privacy. Furthermore, current watermarking algorithms struggle with text quality, information capacity, and robustness, making it challenging to meet the diverse identification needs of LLMs. Thus, we propose a novel multi-bit watermarking algorithm and an open-source toolkit to facilitate research. Experiments show our CredID enhances watermark credibility and efficiency without compromising text quality. Additionally, we successfully utilized this framework to achieve highly accurate identification among multiple LLM vendors.
1. Introduction
As Large Language Models (LLMs) become ubiquitous, the need to identify AI-generated content is critical for copyright protection, security, and legal compliance. However, current watermarking techniques face three major hurdles: a lack of credibility due to "private" verification schemes, low information capacity (often limited to 1-bit detection), and a detrimental trade-off between watermark robustness and text quality (perplexity).
CredID addresses these gaps by creating a collaborative, verifiable, and high-capacity watermarking ecosystem. By introducing a Trusted Third Party (TTP), the framework moves watermarking from a proprietary feature to a verifiable standard, essential for the legal validity of AI identification.
2. The CredID Framework
CredID splits the watermarking process into a collaborative architecture between a Trusted Third Party (TTP) and LLM Vendors. This multi-party approach ensures that no single entity has total control, enhancing the credibility of the identification process.
In the embedding phase, vendors request a unique "watermark seed" from the TTP. Crucially, the vendor does not send the user's prompt to the TTP, preserving user privacy. During extraction, the TTP coordinates with multiple vendors. Each vendor assesses the confidence level that a piece of text contains their specific watermark, and the TTP uses a multi-party voting mechanism to identify the true origin.
3. Methodology: Multi-bit Embedding and Entropy-Aware Selection
The core of CredID is a novel logit-based modification algorithm. It utilizes a "watermark temperature" parameter to control the variance of logit distributions across different messages, allowing the system to embed high-capacity multi-bit identifiers (e.g., a 20-bit ID) into the text.
To maintain linguistic naturalness, the algorithm employs entropy-aware selection. It bypasses "low-entropy" tokens—where the next word is highly predictable—and only applies biases where the model has sufficient "room" to choose without sounding unnatural. This strategy ensures that the mean perplexity increase remains minimal, typically less than 10%.
4. Experimental Evaluation
The researchers evaluated CredID using several diverse datasets, including C4 News, OpenGen (WikiText-103), and student essays from the IvyPanda database. Human-written text was used as a control group to test for false positives. The framework was tested across multiple state-of-the-art models, including LLaMA-2 (7B/70B), LLaMA-3 (8B), Gemma-7B, Falcon-7B, and Qwen-72B.
5. Results and Discussion
Experimental results demonstrate that CredID significantly outperforms existing watermarking methods in balancing success rate and text quality.
As shown in Figure 4, CredID maintains a ~95% success rate for watermark extraction at 0.1 BPW. Even at a higher capacity of 0.25 BPW, it remains highly effective. The watermark also proved remarkably robust, surviving 50% deletion attacks, word substitution attacks, and "copy-paste" attacks where AI text is mixed with human writing. Furthermore, testing on benchmarks like MMLU and GSM8K showed that the watermark had a negligible effect on the LLM’s reasoning and knowledge-retrieval abilities.
6. Conclusion
CredID represents a major step forward in LLM identification by providing a credible, multi-party foundation. By combining a privacy-preserving TTP architecture with a high-capacity, entropy-aware embedding algorithm, it offers a robust solution for identifying AI-generated content without compromising performance. The authors have released an open-source toolkit to encourage further research and adoption of this verifiable standard.
Summary updated