Fine-grained Visible Watermark Removal

Li Niu, Xing Zhao, Bo Zhang, Liqing ZhangICCV 2023 · DOI

Introduction

Visible watermarks are a standard tool for copyright protection, yet their removal remains a challenging inverse problem in computer vision. The difficulty stems from the extreme variability in watermark characteristics, including position, geometry, color, and transparency. While previous state-of-the-art methods employed image-specific dynamic networks to adapt to these variations, they often treated the entire watermarked region as a single, monolithic entity. This "blob-based" approach frequently fails when a watermark consists of multiple components with distinct visual properties—such as a vibrant logo paired with delicate white text—resulting in "ghosting" artifacts or incomplete restoration.

This paper introduces a novel fine-grained approach that shifts the focus from image-specific adaptation to part-specific adaptation. By decomposing a watermark into its constituent parts, the proposed model can apply specialized restoration techniques to each component, significantly improving the quality of the recovered background.

Methodology: Query-Based Part Decomposition

The core of the proposed method is a query-based multi-task framework designed to identify and restore individual watermark components. The architecture consists of a shared encoder and two specialized decoders for mask prediction and background restoration.

Visual examples of watermarked images, their corresponding masks, and the isolated watermarks for two different datasets.

1. Part Query Embeddings and Transformer Adaptation

The model utilizes a set of $K$ learnable "queries"—vectors that encode prior knowledge about common watermark characteristics like specific color ranges or geometric patterns. These queries interact with the input image features through a Transformer block, allowing them to adapt dynamically to the specific watermark parts present in a given image.

2. Fine-grained Mask Prediction

The Mask Decoder uses these adapted queries to generate $K$ distinct "part masks." To prevent the model from collapsing into redundant predictions, the authors implement a low-entropy loss. This constraint forces the network to assign each pixel to a specific part, effectively performing a visual decomposition of the watermark.

3. Part-aware Feature Modulation (PFM)

For restoration, the Background Decoder employs Part-aware Feature Modulation. The adapted queries generate specific convolution kernels for each identified part. These kernels are applied selectively to their corresponding regions (as defined by the part masks), enabling the model to use different mathematical operations for restoring text versus graphical logos.

Experimental Results

The model was rigorously evaluated on two major benchmarks: the Colored Large-scale Watermark Dataset (CLWD) and LOGO30K.

  • Quantitative Performance: The proposed method achieved state-of-the-art results across all metrics. On the CLWD dataset, it reached a Peak Signal-to-Noise Ratio (PSNR) of 39.45, outperforming the previous best method (DKSP, 38.84). Localization accuracy also improved, with the Intersection over Union (IoU) rising to 0.7909 from 0.7730.
  • Qualitative Analysis: Visual comparisons demonstrate that the fine-grained approach effectively eliminates the "shadows" and blurry traces common in previous methods. By treating the watermark as a composite of parts, the model produces backgrounds that are nearly indistinguishable from the ground truth, even in complex scenarios involving multi-colored logos and intricate text.

Conclusion and Future Directions

By moving beyond monolithic watermark processing, this research demonstrates that decomposition is key to high-fidelity image restoration. The use of queries as "specialists" for different visual patterns allows for a more nuanced and effective removal process.

Future work may address current limitations, such as the fixed number of parts ($K$) and the computational overhead introduced by Transformer blocks and dynamic kernels. Additionally, extending the model's robustness to "in-the-wild" watermarks—which may include non-linear distortions or physical artifacts—remains an important next step for practical deployment.

Summary updated

BibTeX
@inproceedings{niu2023fine,
  author = {Li Niu and Xing Zhao and Bo Zhang and Liqing Zhang},
  title = {{Fine-grained Visible Watermark Removal}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01173},
}
Figure