TL;DR
Bypassing tokenizers doesn't solve the cross-lingual fairness gap in AI watermarking; it just moves the problem. This paper reveals that "edit-based" watermarks (using homoglyphs and invisible characters) vary in capacity by over 3x across languages because the Unicode standard provides an uneven supply of "hidden" characters, leaving lower-resourced languages with significantly weaker protection.
What Problem Does This Paper Address?
As AI-generated text becomes ubiquitous, watermarking is essential for identifying its provenance. Most current "generation-time" watermarks work by biasing how a model samples tokens. However, these methods fail in low-resource languages because tokenizers—the tools that chop text into pieces for models—treat different languages unequally, splitting low-resource text into more, smaller fragments that dilute the watermark signal.
"Edit-based" watermarks seemed like a promising, fairer alternative. These methods don't touch the model or the tokenizer; instead, they hide signals in finished text by swapping characters for homoglyphs (characters that look identical but have different code points, like a Latin 'a' vs. a Cyrillic 'а') or inserting invisible characters (like zero-width spaces). Because they operate at the character level, researchers hoped they would escape the "tokenizer gap." This paper investigates whether that hope is justified.
Methods
The authors developed a comprehensive framework to measure the "capacity" of 40 language varieties across 24 scripts.
- Capacity Metric: They defined capacity as the average number of hidden bits a single character can carry. This is split into two tracks: substitution (swapping for homoglyphs) and insertion (adding invisible characters).
- Transformation Ladder: They tested how well these watermarks survive various "attacks," ranging from routine handling (copy-paste, Unicode normalization) to deliberate removal attempts using Unicode's own security defenses (like "skeleton mapping," which collapses homoglyphs into a single form).
- Keyed Detector: To validate their theoretical findings, they built a real-world detector to measure exactly how much text is needed to reliably identify a watermark in different languages under various conditions.
Datasets
- FLORES+: A high-quality human-translated parallel corpus. The authors used the devtest split (1,012 sentences per language) to ensure that differences in capacity were due to script and spelling, not content.
- Unicode 17.0.0 & CLDR 48.2.0: Used as the ground truth for character properties, homoglyph (confusable) lists, and language-specific exemplar characters.
Key Insights and Ideas
- The Gap Just Moved: Removing the tokenizer didn't remove the inequity. Instead of a "tokenizer gap," we now have a "Unicode gap." The capacity to hide information is a property of the script itself.
- Script is Destiny: Script accounts for a staggering 96% of the variation in watermarking capacity. Latin-script languages are "rich" in homoglyphs, while many South Asian and South-East Asian scripts are "poor."
- The Double Penalty for Low-Resource Languages: Languages written in "homoglyph-poor" scripts (like Lao or Khmer) rely heavily on invisible characters for their capacity. However, routine operations like copy-pasting often strip these invisible characters.
- Resource Level vs. Script: Within a single script (like Latin), a language's resource level (how much data is available for it) doesn't affect capacity. However, across the globe, lower-resourced languages tend to use scripts that Unicode has historically supported less comprehensively with homoglyph data.
Key Results
- 3.3-fold Capacity Range: Total capacity ranges from 7.09 bits per character (English, Swahili) down to 2.14 bits (Khmer).
- Copy-Paste Widens the Gap: When routine copy-pasting removes invisible characters, the disparity jumps from 3.3-fold to 8.5-fold.
- Detection Length Disparity: To detect a watermark after a copy-paste, a detector needs only about 2 characters of English or Persian text, but requires 93 characters of Lao.
- Fairness Correlation: There is a moderate correlation ($\rho = +0.54$) between a script's resource level and its watermarking capacity, confirming that lower-resourced communities receive less protection from these tools.
Limitations and Future Work
- Font Relativity: Perceptual "invisibility" is not absolute; it depends entirely on the font being used. A homoglyph that is identical in one font might look jarringly different in another.
- Corpus Constraints: The study relies on the FLORES+ parallel corpus, which represents a specific register of language. Capacity might vary in more informal or specialized text.
- Future Directions: The authors suggest that narrowing this gap requires more complete "confusables" data for under-served scripts in the Unicode standard, or the development of new watermarking channels that don't rely on script-specific inventories.
Summary updated