TL;DR
TRACEMARK is an "editor-aware" watermarking framework that enables user attribution in text-guided image editing by training watermark components directly through a fixed generative editor. By optimizing for user identification rather than just message recovery, it achieves high attribution accuracy (up to 87.5% for 500 users) even after significant semantic changes to the image content.
What Problem Does This Paper Address?
When a user requests an edit from a text-guided image editing service (like "make the hair blond"), the service records the request. However, once the edited image is downloaded and shared, that link to the requesting user is lost.
Traditional watermarking often fails here because generative editors don't just add noise—they reconstruct image content. This "semantic editing" can easily wipe out watermarks designed to survive simple distortions like JPEG compression or resizing. The challenge is to embed a signal before the edit that is robust enough to survive the generative process while remaining invisible and preserving the quality of the requested edit.
Methods
The authors propose TRACEMARK, which shifts the focus from recovering a binary message to directly identifying a user from a registry.
The framework consists of three main components trained jointly:
- User-Conditioned Encoder ($E_\theta$): Embeds a user-specific signal into the original image based on a learned user embedding.
- Shared Detector ($D_\phi$): Maps the edited image into a feature vector (embedding).
- User Prototypes ($p_u$): A bank of learned reference vectors in the embedding space, one for each registered user.
The "secret sauce" is the editor-aware training loop. The encoder and detector are trained through a fixed-weight generative editor (like InstructPix2Pix). Gradients from the identification error are propagated back through the editor to the encoder, forcing it to learn watermarks that the specific editor is likely to preserve.
The loss function balances four objectives:
- Carrier Fidelity: Keeps the watermarked input looking like the original.
- Edit Fidelity: Ensures the watermarked edit looks like the intended unwatermarked edit.
- Embedding Consistency: Encourages the detector to produce similar embeddings for an image before and after editing.
- User Identification: A cross-entropy loss that maximizes the cosine similarity between the detector's output and the correct user's prototype.
Datasets
The authors evaluated TRACEMARK across three distinct image domains, using 30 editing instructions per domain (90 tasks total):
- CelebA-HQ: High-quality human faces.
- AFHQ-Dog: Animal faces (specifically dogs).
- CUB-200-2011: Fine-grained bird species.
Training pools for these datasets contained approximately 28,000, 4,700, and 6,000 images, respectively.
Key Insights and Ideas
- Editor Compatibility Matters: A watermark's survival depends heavily on the "family" of the editor. A watermark trained on InstructPix2Pix (based on Stable Diffusion 1.5) performs poorly when the image is edited by SDXL-Turbo, even if the magnitude of the visual change is identical.
- Prototypes > Bitstrings: Moving away from binary message decoding to "prototype matching" (comparing image features to learned user vectors) simplifies the system and improves robustness. It allows the model to learn the most resilient features for a specific editing task.
- The "Capacity Margin": The authors introduce a diagnostic measure to show why standard watermarks fail. Even if a watermark has a high "margin" (surplus information) after JPEG compression, semantic editing can push that margin into the negative, making unique user identification impossible.
Key Results
- High Scalability: TRACEMARK achieved 87.5% mean identification accuracy in a registry of 500 users.
- Superior Robustness: At a common 256-pixel resolution, TRACEMARK significantly outperformed existing baselines (like RoSteALS and VINE-R) on "unseen" editing channels—editors it wasn't specifically trained on.
- Cross-Instruction Reuse: While models are trained on specific instructions (e.g., "add glasses"), they often generalize well to related instructions within the same domain without retraining, maintaining a mean accuracy of 63.5% across 24 test pairs.
Limitations and Future Work
- Task Specificity: Currently, a separate model is trained for each image domain and editing instruction. While some reuse is possible, a more "generalist" watermarking model would be more practical for large-scale services.
- Resolution Sensitivity: The current implementation is native to 256x256 pixels. Baselines like VINE-R still perform better at their native 512x512 resolution, suggesting that TRACEMARK's performance is tied to its training resolution.
- Adversarial Attacks: The paper focuses on attribution under requested edits in a trusted service. It does not evaluate resistance to deliberate, malicious attempts to remove the watermark (adversarial removal).
- ChordEdit Failure: The system struggled with the "ChordEdit" editor, which uses a different underlying transport mechanism, highlighting that there are still limits to cross-editor generalization.
Summary updated