Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking

Liyan Xie, Muhammad Siddeek, Mohamed Seif, Andrea J. Goldsmith, Mengdi Wang

Abstract

Watermarking has become a key technique for proprietary language models, enabling the distinction between AI-generated and human-written text. However, in many real-world scenarios, LLM-generated content may undergo post-generation edits, such as human revisions or even spoofing attacks, making it critical to detect and localize such modifications. In this work, we introduce a new task: detecting post-generation edits locally made to watermarked LLM outputs. To this end, we propose a combinatorial pattern-based watermarking framework, which partitions the vocabulary into disjoint subsets and embeds the watermark by enforcing a deterministic combinatorial pattern over these subsets during generation. We accompany the combinatorial watermark with a global statistic that can be used to detect the watermark. Furthermore, we design lightweight local statistics to flag and localize potential edits. We introduce two task-specific evaluation metrics, Type-I error rate and detection accuracy, and evaluate our method on open-source LLMs across a variety of editing scenarios, demonstrating strong empirical performance in edit localization.
BibTeX
@misc{xie2025detecting,
  author = {Liyan Xie and Muhammad Siddeek and Mohamed Seif and Andrea J. Goldsmith and Mengdi Wang},
  title = {{Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking}},
  year = {2025},
  eprint = {2510.01637},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01637},
}
Figure