Cross-Attention watermarking of Large Language Models

Folco Bertini Baldassini, Huy H. Nguyen, Ching-Chung Chang, Isao Echizen

As Large Language Models (LLMs) become increasingly adept at mimicking human prose, the line between organic and synthetic content has blurred. Traditional "post-hoc" detectors—tools that attempt to identify AI-generated text after its creation—are often unreliable. In "Cross-Attention Watermarking of Large Language Models," researchers propose a proactive solution: embedding a digital signature directly into the model's generation process using cross-attention mechanisms.

The Provenance Challenge

The core issue addressed is provenance. Proving text origin is vital for preventing "model collapse" (where models are trained on their own synthetic data), protecting intellectual property, and mitigating AI-generated misinformation. Existing watermarking methods typically face a "trilemma":

  1. Robustness: Survival against paraphrasing or minor edits.
  2. Quality: Maintaining linguistic coherence and naturalness.
  3. Efficiency: Minimizing impact on inference speed and model size.

Current methods often rely on "green list/red list" approaches that tweak token probabilities. However, these can be random and lack the capacity for complex message embedding.

Proposed Architectures

The researchers leverage cross-attention—a mechanism typically used in multimodal learning to align different data types (e.g., images and text)—to inject watermark bits into the generation process.

Two primary strategies were explored:

1. Gated Cross-Attention Layer

Inspired by the Flamingo model, this approach inserts a lightweight layer between existing decoder blocks. It treats the watermark bits as the "Key" and "Value," while the text context acts as the "Query." A "gating" mechanism, using zero-initialized parameters, ensures that the watermark does not disrupt model performance at the start of training.

2. Decoder Layer Substitution

This more aggressive approach replaces the final layer of a pretrained model with a custom layer designed to handle both self-attention (for text coherence) and cross-attention (for the watermark).

To ensure robustness, a Watermark Extractor (a bidirectional transformer) is trained alongside the generator. The system uses the Gumbel-Softmax trick to bridge the gap between discrete text tokens and differentiable neural networks, allowing the extractor to "read" bits even from slightly modified text.

Key Insights and Results

The study yielded several critical findings regarding the trade-offs in neural watermarking:

  • Multimodal Inspiration: Treating a watermark as a secondary data modality allows the model to naturally weave information into the linguistic structure.
  • The Entropy Constraint: "High entropy" sentences, where many word choices are plausible, are easier to watermark. In contrast, highly predictable "low entropy" text (e.g., factual statements) offers little room for hidden data without degrading quality.
  • Performance Metrics: The gated cross-attention approach achieved approximately 70% bit accuracy while maintaining high text quality. While the substitution method offered higher data capacity, it suffered from quality deterioration in longer sequences (64–128 tokens).
  • Resilience: The method outperformed the Adversarial Watermarking Transformer (AWT) in both text quality (cross-entropy) and watermark reconstruction accuracy. Training with simulated noise significantly improved the extractor's resilience to paraphrasing.

Limitations and Future Directions

Despite the promising results, challenges remain. The substitution model's instability over long sequences suggests a "drift" where the watermark eventually interferes with long-term coherence. Additionally, watermarking short or factual text remains difficult due to the "entropy wall." While the gated approach is parameter-efficient, adding hundreds of millions of parameters to a 7B model still presents a computational hurdle for some deployments. Future work will focus on refining these layers for longer sequences and achieving the "sweet spot" of imperceptible yet unremovable watermarks.

Summary updated

BibTeX
@inproceedings{baldassini2024cross,
  author = {Folco Bertini Baldassini and Huy H. Nguyen and Ching-Chung Chang and Isao Echizen},
  title = {{Cross-Attention watermarking of Large Language Models}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  doi = {10.1109/icassp48485.2024.10446397},
}
Figure