Watermarks That Hide Where Humans Are Blind, Not Where the Loss Is Blind

Learned image watermarking systems are trained with two networks. One hides a message in an image, the other reads it back. What they learn depends mostly on the "invisibility" loss that penalizes changes to the image. If that loss is a deep perceptual metric such as LPIPS, which scores image similarity by comparing features inside a pretrained network, then the hiding network is optimizing directly against a known, differentiable judge. It will put its signal wherever that judge is least sensitive. Those places are not necessarily where people are least sensitive. They also tend to be spectrally unnatural, and spectrally unnatural signals are exactly what modern removal attacks strip. Our claim is that this single mechanism explains two separate complaints in the watermarking literature: residual visible artifacts, and fragility to frequency-based removal. We also claim a structural fix handles both better than any choice of loss.

The fix is to stop asking a loss to enforce invisibility and to build invisibility into how the signal is written. We decompose the image with a wavelet transform, which splits it into frequency bands that keep spatial locality. The watermark is written multiplicatively: each detail coefficient $c$ becomes $c\,(1+\rho_b\, t)$, where $t\in[-1,1]$ is the encoder's output at that location and $\rho_b$ is a small capped strength for band $b$. This one change has three consequences. Embedding strength becomes content-adaptive automatically, since smooth regions have near-zero coefficients and receive nothing. The band energy of the watermarked image stays within a factor $1+\rho_b^2$ of the host, so the spectrum keeps the host's natural fall-off. And because detail coefficients have random signs across images, averaging many watermarked images cancels the mark rather than revealing it.

Core Claim

We claim three things. Encoders trained against learned perceptual losses exploit those losses, and the exploitation is measurable as a shift of watermark energy into frequency bands where the host has little energy. A multiplicative, band-capped wavelet parameterization removes this failure. At matched robustness it will be less visible to humans than LPIPS- or BiRQA-trained encoders, it will be harder to strip with spectral projection attacks, and it will survive multi-image averaging. It will also do this in one forward pass, where the strongest adaptive competitor needs a per-image search taking minutes.

How This Relates to Prior Work

Three papers motivate the proposal by documenting the failures we target. Can Simple Averaging Defeat Modern Watermarks? shows that watermarks with a fixed pattern are exposed by averaging 50–100 images: Tree-Ring's detection AUC drops from 1.0 to about 0.24. Content-adaptive learned schemes such as HiDDeN and RivaGAN keep detection above 0.95. DAWN removes watermarks by projecting 8×8 block-DCT spectra back onto natural power-law decay and then running a single diffusion pass. It reports over 95% removal of classical schemes, including RivaGAN, and 60–90% removal of semantic schemes. Together these results say that content-adaptive learned marks survive averaging but break under spectral projection, while structured spectral marks show the reverse pattern. Our proposal sits between them: the mark is content-adaptive and also spectrally natural. The third motivating paper is BiRQA. It shows that under white-box attack on KADID-10k, undefended neural quality metrics fall to rank correlation with human scores (SROCC) of 0.30–0.57. That is direct evidence that an optimizer can push learned metrics far from human judgment. A watermark encoder trained with gradients from such a metric is exactly that kind of optimizer.

Several papers already combine adaptive strength with frequency information or with perceptual metrics, and we differ from each concretely. ZoDiac writes Fourier rings into an inverted diffusion latent and binary-searches a blend weight per image. It achieves strong robustness but carries no message beyond presence (zero-bit) and costs 45–255 seconds per image. We carry a multi-bit payload and set strength in closed form. PiGW and Tree-Ring mask frequency bands in the initial noise, which makes the pattern identical across images and hence averageable. Our pattern changes sign with the host. Dynamic watermarks in images generated by diffusion models scales a pixel-space mark by SSIM and semantic similarity, and SIREN shapes perturbations with a CIEDE2000 color-difference constraint. Both adapt strength through a loss or a metric in pixel or color space. We argue that is the step that invites exploitation.

CompMarkGS uses Fourier analysis to decide where to add capacity in 3D scenes. It is evaluated only with PSNR and bit accuracy, so it does not test the human-versus-metric question. An Adaptive and Secure Holographic Image Watermarking Scheme sets strength from entropy in contourlet bands with hand-designed rules and no learned decoder or attack model. Our parameterization borrows its spirit, meaning strength that follows local band content, and borrows the multiplicative form from classical spread-spectrum watermarking. What is new is putting that form inside a learned encoder–decoder trained against modern removal attacks, and testing it against learned perceptual losses on human judgments.

Two frequency-supervision papers supply method rather than motivation. Joint Sub-bands Learning with Clique Structures for Wavelet Domain Super-Resolution shows that per-sub-band normalization keeps low frequencies from dominating gradients. We use the same normalization so the decoder's read-out loss is not swamped by the coarse band. Coloring the Noise (ASASR) frames the gap between metric scores and perceived quality in super-resolution as a spectral misalignment between the training objective and natural image statistics. This is the diagnosis we transfer to embedding. We deliberately do not import its Sobolev adversarial machinery: it is expensive, and its regularity index is hand-fixed at $s=1.5$. Finally, LPIPS matters here both as the loss we expect to be exploited and as the reason not to dismiss deep metrics. Its 2AFC agreement with humans exceeds classical metrics on BAPPS, against a human-to-human ceiling of 73.9%. Our claim is narrower than "LPIPS is bad." It is that LPIPS measures well when it is only used to score images and becomes a weak constraint once an encoder is optimized directly against it.

Research Program

The work has a diagnostic part and a constructive part.

The diagnostic study trains one encoder–decoder architecture, HiDDeN-style with a fixed differentiable attack layer, under four invisibility losses: pixel MSE, LPIPS, BiRQA, and an explicit FFT amplitude-matching loss. Everything else is held fixed. For each trained model we compute the residual (watermarked minus host) and measure its energy per wavelet band relative to the host's energy in that band. We call this the envelope ratio. We predict that metric-trained encoders put a disproportionate share of residual energy in bands and regions where the host has little energy. We then measure an exploitation gap: how much each encoder's training metric overstates quality, relative to held-out metrics and to human 2AFC judgments. Finally we check whether the envelope ratio predicts both human visibility and DAWN removal rate across models. If it does, one spectral quantity explains both failures.

The constructive part is the multiplicative wavelet encoder. The encoder sees the image and the message. It outputs a bounded field $t_{b,x}$ over the detail sub-bands of a two- or three-level wavelet decomposition. The watermarked coefficients are $c_{b,x}(1+\rho_b t_{b,x})$, and the coarse band is left untouched to avoid visible color and luminance shifts. Each $\rho_b$ is learned but clipped to a ceiling $\bar\rho_b$. The ceiling is set once from a just-noticeable-difference calibration on BAPPS-style JND judgments, not tuned per image. The decoder reads a normalized sub-band representation.

Training uses only a message-recovery loss plus an attack layer. The attack layer covers JPEG, resizing, noise, a VAE round-trip, a single low-strength diffusion regeneration pass, and a differentiable surrogate of DAWN's block-DCT spectral projection. No perceptual loss is used at all. This is deliberate: it tests the claim that the parameterization, not a loss, carries the invisibility.

We will be explicit about two costs of this design. Smooth, low-texture images offer little capacity, so we will report bit accuracy as a function of image wavelet energy. Wavelets are also not rotation-invariant, and we will report rotation as an acknowledged weakness rather than hide it.

Baselines

The do-nothing reference points are the unwatermarked image and a uniform-strength additive encoder trained with MSE. These anchor human 2AFC at chance and give the cost of ignoring content.

The main learned competitor is the same encoder–decoder trained with LPIPS as its invisibility loss. This is what a reasonable engineer would build today, and it isolates the effect of optimizing against a learned metric. A second, stronger learned competitor uses BiRQA as the loss. It was designed to resist adversarial manipulation, so it is the fairest test of whether a better metric is enough.

A third competitor adds an FFT amplitude-matching loss to the LPIPS encoder. This tests whether a soft spectral penalty achieves what our hard parameterization does.

The strongest existing system for robustness is ZoDiac. It reports detection rates above 98% against Stable-Diffusion-based removal, where baselines fell to 0–48%, with PSNR around 30 dB and SSIM of 0.92, at 45–255 seconds per image. Because ZoDiac is zero-bit, we compare it on detection at a fixed false-positive rate and treat our multi-bit decoding as a detector at the same false-positive rate. We include Tree-Ring for the averaging protocol, where its reported AUC after averaging is about 0.24. HiDDeN and RivaGAN serve as the content-adaptive reference, reported above 0.95 after averaging but above 95% removed by DAWN.

Fairness requires matched robustness. Every learned method gets the same training data (MS-COCO 2017 train), the same attack layer, the same architecture backbone, the same GPU-hour budget, and the same hyperparameter-search budget. Each method's strength knob is then swept until it reaches the same bit accuracy under a fixed held-out attack suite. Visibility is compared only at that matched operating point.

Two ablations isolate the key ingredient. The first is additive-proportional embedding, $c+\rho_b |c|\, t$: it keeps content-adaptive magnitude but removes the host-sign dependence, and it should collapse under averaging. The second is multiplicative embedding with the band caps removed, which shows what the JND ceilings contribute.

Beating the field means three things at matched robustness. First, lower human 2AFC detection of the watermark than both the LPIPS and BiRQA encoders. Second, a lower DAWN removal rate than the HiDDeN/RivaGAN level of over 95%. Third, averaging-attack detection at least at the above-0.95 level reported for content-adaptive schemes. All of this with a payload and single-pass latency ZoDiac does not offer.

Why not just…?

Why not just train with a more robust perceptual metric such as BiRQA? This is the most serious threat. BiRQA's adversarial training lifted white-box SROCC on KADID-10k to 0.60–0.84, so an encoder may find it much harder to exploit. But 0.60–0.84 is still well below the metric's clean correlation, which leaves room for an optimizer to find blind spots. BiRQA's analytic features also say nothing about spectral naturalness, so the encoder may still produce marks DAWN can find. The BiRQA-loss baseline settles this directly. If it matches our method on human 2AFC and on DAWN removal at matched robustness, then the structural constraint is unnecessary, and we will report the proposal as refuted in favor of better losses.

Why not just add a frequency-domain loss that penalizes spectral deviation? A soft spectral penalty is still a differentiable judge. The encoder can satisfy a global amplitude-matching term while concentrating energy locally, because global Fourier losses ignore where energy sits, a limitation noted across the frequency-supervision literature. The multiplicative form ties the mark to local host energy at every location by construction. The FFT-loss baseline measures whether the soft version closes the gap, both on the envelope ratio and on human visibility.

Why not just use a classical fixed mid-frequency DCT watermark with perceptual masking? Classical frequency-domain schemes are already known to fail against modern pipelines. ZoDiac reports traditional baselines at 0–48% detection under diffusion regeneration and 0% under composite attacks, and DAWN removes DWTDCT-style marks over 95% of the time. Our method keeps the masking intuition but adds a learned decoder and training against regeneration and projection. That is where the classical methods break.

Why not just use ZoDiac, which is already adaptive and robust? ZoDiac is the robustness bar we must approach, and we may not beat it on raw detection under composite attacks. But it is zero-bit, it needs minutes per image, and its rings are the structured spectral anomaly that DAWN's projection targets. We will report whether we match its detection at a fixed false-positive rate while carrying a payload in one forward pass. Falling short on detection while winning on payload and speed would be a real trade-off, not a win, and we will present it as such.

Proposed Experiments

We evaluate on MS-COCO 2017 validation images and on DiffusionDB generations, following the protocols of Can Simple Averaging Defeat Modern Watermarks?, DAWN, and ZoDiac.

The robustness suite has four parts: standard distortions (JPEG, resize, crop, noise, blur), diffusion regeneration, DAWN, and the averaging attack over 100 images carrying the same message. We also run the beige-box VAE fine-tuning attack from the first-place NeurIPS 2024 removal solution, which reported a 95.7% average removal rate.

Quality is measured four ways: PSNR, LPIPS, BiRQA, and FID computed by the standard protocol, because the non-standard FID scale in the dynamic-watermark paper cannot be compared. Most important is a human 2AFC study. Participants see clean and watermarked crops at the matched robustness point and pick which one is altered, with a JND variant. The 73.9% inter-human agreement from LPIPS gives a reference for noise in these judgments. Latency is measured per image on a single GPU against ZoDiac's reported 45–255 seconds.

Four decisive checks determine the outcome. First, the envelope ratio must separate metric-trained encoders from ours and must correlate with both human detection and DAWN removal across all trained models. If it predicts neither, the proposed mechanism is wrong. Second, the multiplicative encoder must beat the LPIPS and BiRQA encoders on human 2AFC at matched robustness. Third, the averaging attack must leave the multiplicative method intact while the additive-proportional ablation collapses toward Tree-Ring's behavior. This confirms that sign cancellation, not magnitude adaptation alone, provides averaging resistance. Fourth, DAWN removal against our method must fall clearly below the over-95% rate it achieves on learned content-adaptive marks. If DAWN's success rate stays high, then keeping the spectrum natural does not protect a mark once an attacker projects back to natural decay.