Paraphrase-Resilient Text Watermarking via Semantic Embeddings

126 concepts65 papers1 proposal

As large language models (LLMs) become increasingly capable of generating human-quality text, the ability to identify machine-generated content has become a critical requirement for digital accountability. One of the most prominent solutions is text watermarking, a technique that subtly biases an LLM's token selection during generation to leave a predictable statistical signature.

However, traditional watermarking algorithms suffer from a fundamental flaw: they are incredibly fragile. If an adversary takes watermarked text and runs it through a paraphrasing tool, swaps a few words for synonyms, or alters the sentence structure, the watermark is completely erased.

To solve this vulnerability, a new research direction has emerged: semantic watermarking. Instead of tying the watermark to fragile, surface-level token sequences, these techniques anchor the watermark to the underlying meaning of the text. Because paraphrasing alters syntax but preserves semantics, these watermarks remain robustly detectable even after heavy rewriting.


The Fragility of Token-Level Hashes

To understand why semantic watermarking works, we must first understand why traditional watermarking fails.

Standard watermarking algorithms operate auto-regressively at the token level. To determine which tokens are preferred (the "green list") at step $t$, the algorithm computes a deterministic cryptographic hash of the preceding tokens (e.g., tokens $t-1$ or $t-2$). This hash acts as a random seed to shuffle the vocabulary.

Traditional Token-Based Watermarking:
[The] -> [quick] -> [brown] ---> Hash("quick brown") ---> Seed 42 ---> Green List: {fox, dog, cat}
                                                                        Actual Output: "fox"

After Paraphrasing Attack:
[The] -> [speedy] -> [auburn] -> Hash("speedy auburn") -> Seed 99 ---> Green List: {wolf, bear, lion}
                                                                        (Watermark Signature Lost!)

If an attacker changes even a single word—replacing "quick brown" with "speedy auburn"—the cryptographic hash changes completely. The pseudo-random number generator yields an entirely different seed, generating a completely unrelated green list. The statistical alignment is broken, and the watermark is effectively washed away.


The Core Insight: Semantic Invariance

Semantic watermarking replaces discrete token hashes with continuous semantic embeddings.

When text is passed through a dense encoder model, sentences with similar meanings are mapped to nearby coordinates in a high-dimensional vector space. As shown by single-author work from Rush, these contextual embeddings naturally preserve semantic intent across diverse phrasings and grammatical variations.

By using these stable vector representations as the context or "seed" for the watermark, the generation process becomes invariant to surface-level lexical changes. If "The quick brown fox" and "The speedy auburn fox" map to nearly identical vectors, they can be made to trigger the same watermarking rules.

Semantic Watermarking:
[The quick brown]  ---> Semantic Encoder ---> Vector [0.82, -0.11] ---\
                                                                       ---> Same Stable Key / Green List
[The speedy auburn] ---> Semantic Encoder ---> Vector [0.81, -0.13] ---/

Mechanism 1: Discretizing the Continuous Space

The immediate engineering challenge of semantic watermarking is turning a continuous vector into a discrete key or green list. Because embeddings drift slightly when text is paraphrased, a naive rounding mechanism might push a vector across a boundary, destroying the key.

To solve this, Guo et al. utilize Locally Sensitive Hashing (LSH). LSH is a mathematical technique designed to map high-dimensional vectors into discrete hash buckets such that nearby vectors share the same bucket with high probability. By passing context embeddings through an LSH function, the model generates stable, low-dimensional keys that absorb minor variations in word choice.

Complementing this, Xu et al. introduced SemaMark, which uses a weighted pooling of recent token embeddings to maintain a stable context. To handle the inevitable boundary cases where an embedding does cross a discretization threshold, the authors implement a fuzzy detection strategy. During verification, the detector evaluates tokens against a search window of neighboring regions in the discretized semantic space, compensating for semantic drift.


Mechanism 2: Semantic Key Retrieval

Instead of computing a hash on the fly during generation and detection, an alternative paradigm relies on indexing keys within a database.

In single-author work, Huang developed WaterPool, a framework that decouples key generation from local token sequences entirely. During generation, the system uses the semantic embedding of the text to retrieve a specific secret key from a database. During detection, the candidate text's embedding is used as a search query. Because the retrieval relies on semantic similarity rather than exact character matches, the correct key is successfully pulled even if the text has been heavily paraphrased or translated.

This aligns with broader findings by Krishna et al., who demonstrated that while surface-level statistical signatures are easily scrubbed by rewriting, the core semantic content remains highly stable. The authors showed that maintaining a database of all generated outputs and using semantic retrieval serves as an incredibly robust defense against heavy paraphrasing attacks.


Mechanism 3: Multi-Channel and Sentence-Level Distribution

Rather than embedding a watermark token-by-token, several recent frameworks operate at the sentence level to distribute the signature across broader linguistic structures.

Huo et al. developed PMark, which shifts the watermarking unit from individual tokens to entire sentences. The authors embed watermark evidence across multiple orthogonal pivot vectors within a single sentence representation. This creates several independent semantic channels:

$$\text{Sentence Embedding} \cdot \mathbf{v}_{\text{pivot}}^{(i)} \to \text{Channel } i$$

Because the watermark signature is densified and distributed throughout the sentence's high-dimensional semantic space, an attacker cannot easily disrupt the signal through local word-level edits or synonym swaps without fundamentally destroying the sentence's meaning. Similarly, Liu et al. utilize sentence-level embeddings to capture high-level context, ensuring that the watermark pattern remains stable across structural modifications.


Mechanism 4: Lexical Redundancy and Synonym Clustering

Another intuitive approach to semantic invariance operates directly on the model's vocabulary by grouping words with identical meanings.

Chen et al. proposed WatME, which leverages lexical redundancy. Instead of partitioning individual tokens into green and red lists, the framework clusters synonyms into semantic groups. The watermark is embedded by biasing selection toward specific synonym groups rather than individual words. If an adversary paraphrases the text by replacing a word with its synonym, the text remains within the same semantic cluster, preserving the statistical signature required for detection.

WatME Synonym Clustering:
Cluster 1 (Speed): {quick, fast, speedy, rapid} -> Marked as "Green"
Cluster 2 (Color): {brown, auburn, chestnut}     -> Marked as "Red"

Whether the model outputs "quick" or the attacker changes it to "speedy", 
the text still draws from the Green Cluster, preserving the watermark.

The Spoofing Dilemma: Global Semantic Conditioning

While semantic invariance makes watermarks highly resilient to attacks, it introduces a dangerous security vulnerability: spoofing attacks. If a watermark is completely indifferent to minor textual changes, a malicious actor could take a legitimately watermarked AI document, append a paragraph of harmful content (such as hate speech), and the detector might still validate the entire text as watermarked—falsely attributing the harmful content to the model provider.

To counteract this, An et al. introduced Global Semantic Conditioning. Using contrastive representation learning, the authors train a semantic mapping model to enforce a strict boundary condition: the watermark must be invariant to semantic-preserving transformations (like paraphrasing) but highly fragile under semantic-distorting operations (like altering sentiment or adding malicious clauses).

By conditioning the watermark on a global embedding of the entire text, any localized, meaning-distorting edit alters the global signature, causing the watermark detection score to drop immediately and preventing spoofing validation.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 69 papers
  2. Version 1 · 2026-10-04 · 134 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_invariant_watermarking_73bfddc5,
  title = {semantic-invariant watermarking},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-invariant-watermarking/rdr_73bfddc5349d485a891d99ab108478ed},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_semantic_invariant_watermarking_73bfddc5_v2,
  title = {semantic-invariant watermarking},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-invariant-watermarking/rdr_73bfddc5349d485a891d99ab108478ed/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake