Invisible watermarks and hidden data payloads are designed around one rule borrowed from human vision: people barely notice changes in color (chrominance) or in the blue channel, so put the payload energy there and leave brightness (luminance) alone. Images now pass through object detectors and vision-language models far more often than through human eyes, and nobody has checked whether that rule protects those models too. This proposal tests one question directly. At the embedding strength a watermark actually needs in order to survive compression and resizing, does putting the payload in chroma or blue instead of luminance hurt a downstream detector more, less, or the same?
Answering this requires a benchmark protocol that the watermarking field does not have. Papers in the field compare schemes on a single "Lena" image or three legacy video clips, and each scheme is reported at whatever strength its authors chose. The protocol we propose compares every channel choice at a matched operating point on MS-COCO and reports detector accuracy alongside the usual bit error rate and PSNR (peak signal-to-noise ratio, a pixel-error score). It also tests whether a cheap, detector-agnostic feature-distortion score can stand in for running many detectors.
Core Claim
We make two claims. The first is that the ranking of channel allocations by harm to machine vision depends on how schemes are matched. Matching at equal PSNR, the implicit convention in the field, favors chroma and blue embedding. We hypothesize that matching at equal robustness, meaning the same bit error rate after a fixed attack suite, can reverse that ranking. The reason is that chroma and blue payloads must be embedded more strongly to survive chroma subsampling in JPEG and video codecs, and that extra energy lands in features detectors use. The second claim is that the size of the detector accuracy drop, and the ranking of schemes by it, can be predicted without a task head from how far the watermark moves a frozen general-purpose backbone's features. We hypothesize this proxy transfers to held-out detectors about as well as a predictor trained against one detector.
Either claim can fail, and a failure is still informative. If detectors turn out to be insensitive to chroma at every robust operating point, the human-vision rule is also safe for machines, and the field can keep using it with evidence behind it.
Why This Is Not Already Done
The closest work splits into three groups, each missing one piece.
The first group adds human-vision channel choices but never measures independent machine utility. Generative Model Watermarking Based on Human Visual System embeds in R/B or U/V inside a segmentation generator trained on Cityscapes. It reports pixel scores (PSNR 23.12 dB versus 20.83 dB for a uniform baseline), but it never tests off-the-shelf models on the watermarked outputs, and it never separates luminance from chroma effects at matched robustness. Robust Watermarks Leak shows that robust watermarks concentrate in specific channels of a pretrained network's internal features. That finding motivates our hypothesis, but the paper uses it for attacks and does not measure task accuracy.
The second group measures machine utility but not watermarks. Machine Vision-Aware Quality Metrics for Compressed Image and Video Assessment finds that PSNR, SSIM and VMAF correlate with AI performance loss at a Spearman rank correlation (SRCC) of only about 0.24 across codecs. It trains task-specific predictors that reach 0.8–0.9, and a generalized model reaches 0.837. Its degradations are all codec artifacts. Single Image Deraining: A Comprehensive Benchmark Analysis shows that restoration with good pixel scores can lower YOLO-v3 mAP from 23.27 to 20.45. Machine-oriented Visual Media Quality Assessment reports human-oriented metrics correlating below 0.3 with VLM and robot task success. These three papers establish that pixel fidelity is a poor guide for machines. We reuse their measurement logic, and one of them supplies our strongest metric baseline.
The third group covers benchmarking itself. Invisible Image Watermarks Are Provably Removable Using Generative AI evaluates watermarks on 500 COCO images and removes 99% of pixel watermarks while keeping PSNR above 30 dB, but it measures only detection of the mark and not detection of objects. Revisiting Point Cloud Shape Classification shows how an unmatched protocol can manufacture apparent progress: PointNet++ rose from 90.0% to 93.3% on ModelNet40 once training was standardized. We apply that lesson to the free choice of embedding strength in watermarking.
Research Program
The core tool is a controlled family of embedders in which only the target channel varies. We use one transform-domain algorithm, with blockwise mid-frequency DCT coefficients modulated by a spread-spectrum payload, and apply it in turn to Y, U, V, R, G, B and uniform RGB. Channels come from YUV and CIE L*a*b* conversions. Holding the algorithm fixed means channel effects cannot be confused with algorithm effects.
Next to this family we reimplement three published designs. The first is the 2:1:4 R:G:B weighting of A Color Image Watermark Algorithm Based on Logistic and DWT-SVD. The second is blue-channel embedding with a green reference channel from Blind Robust Video Watermarking Based on Adaptive Region Selection and Channel Reference. The third is Y-only DWT embedding in YIQ space, following An optimized digital watermarking algorithm in wavelet domain based on differential evolution. We also run the learned or standard schemes evaluated by Zhao et al., namely DwtDctSvd, RivaGAN, StegaStamp and SSL, to the extent that implementations are available.
For every scheme we sweep embedding strength and fix three matching rules. Under matched PSNR, all schemes sit at 40 dB. Under matched perceptual difference, all schemes have the same mean CIEDE color difference. Under matched robustness, all schemes reach bit error rate ≤1% for a 64-bit payload after a fixed attack suite: JPEG at quality 50 with 4:2:0 chroma subsampling, 0.5× resize, and Gaussian noise. We then measure harm to downstream models on COCO val2017 Microsoft COCO. The primary metric is the change in box mAP averaged over IoU thresholds 0.5–0.95, reported separately for small objects. The detectors are YOLO-v3 as used in MPID, RTMDet-s as used in ReRAW, and a two-stage instance segmentation model. We add Cityscapes semantic segmentation and a VLM question-answering subset scored in the style of MoIQA-Sim.
Finally, we run the generative regeneration attacks from XuandongZhao/WatermarkAttacker on both watermarked and clean images. This measures whether purification itself costs detector accuracy.
For the proxy metric, let $f_\ell$ be layer $\ell$ features of a frozen general backbone that is never used as a detector in evaluation. We define
$$D(x,\tilde x)=\frac{1}{L}\sum_{\ell=1}^{L}\frac{\lVert f_\ell(x)-f_\ell(\tilde x)\rVert_2}{\lVert f_\ell(x)\rVert_2},$$
where $x$ is the clean image, $\tilde x$ the watermarked one, and $L$ the number of layers sampled. We score the proxy by rank correlation against the per-image and per-scheme mAP changes of held-out detectors. This metric is secondary to the benchmark and matters only where running detectors is too costly, such as inside an embedding optimizer.
Baselines
The naive reference point is the unwatermarked image, whose mAP defines zero harm. Alongside it we report PSNR and SSIM ranking, the implicit decision rule the field uses today. The claim that matching changes conclusions is supported if the channel ranking under matched robustness differs significantly from the ranking under matched PSNR. We test this with Kendall's tau on bootstrap resamples of COCO val images, and require that at least one pair of allocations reverses order with a confidence interval that excludes zero mAP difference.
The strongest existing method for predicting machine harm is the task-aware predictor of Dremin et al. The file reports its Delta Object IoU at SRCC 0.8–0.9 and its generalized model at SRCC 0.837, both on codec distortions. On the same codec data, PSNR, SSIM and VMAF reach about 0.24. We retrain the predictor's lightweight backbone on watermark distortions from the same training split and give it the same number of strength sweeps and the same tuning budget as our proxy. Both are then evaluated on detectors neither saw. Our proxy beats it if its held-out-detector SRCC is at least equal within bootstrap error while needing no detector labels, or if it is strictly higher on held-out detector families. No reported result was found on Lacuna for any machine-utility predictor on watermarked images, and none for COCO mAP under channel-allocated watermarks.
For watermark schemes, the strongest human-vision design in the file is the blue/green-reference method of Chang et al., which reports PSNR of 49–53 dB and BER of 1.3–2.2% under camcording on legacy clips. We rerun it on COCO under our protocol instead of quoting those numbers. Two ablations isolate the key ingredient, which is channel choice at matched robustness. The first holds the algorithm and matched robustness fixed and varies only the channel. The second holds the channel fixed and removes chroma subsampling from the attack suite. If the chroma penalty disappears in the second ablation, subsampling is the mechanism.
Why not just…?
Why not just trust PSNR, since 43–53 dB is invisible? The evidence in the file already argues against this. Codec PSNR correlates with machine loss at about 0.24, and deraining raises pixel scores while lowering mAP. Watermarks are structured, not random noise, so whether this failure carries over is exactly what the matched-PSNR arm measures.
Why not just run the detectors and skip any proxy metric? For the benchmark itself we do exactly that, and full inference is the ground truth. The proxy matters only for detector-agnostic ranking and for use inside optimizers. If it transfers no better than simply running one cheap detector, we will report that and drop it. We treat this as a real possibility.
Why not just adapt Dremin et al.'s trained predictor to watermarks? This is the most serious competitor, and it is run as a baseline under an equal budget. It may win on detectors resembling its training head. The decisive test is transfer to held-out detector families and the VLM subset, where a head-free feature distance has a structural advantage but is not guaranteed to win.
Why not just argue detectors ignore color, so chroma embedding is trivially safe? This is the null result we want to be able to confirm. The channel-leakage findings of Ba et al. make it doubtful at robust strengths, and the matched-robustness sweep either confirms it or refutes it directly.
Decisive Checks
The study succeeds under either of two outcomes. In the first, the matched-robustness ranking differs from the matched-PSNR ranking on COCO mAP for at least two detectors, with small-object mAP showing the largest gap. In the second, all channel allocations stay within bootstrap noise of the clean baseline at every robust strength, and the field can adopt the human-vision rule with evidence. A third check is whether regeneration purification lowers mAP on clean images as much as on watermarked ones; if it does, generative removal has a machine-utility cost that no prior evaluation measured. We will release the protocol, strength sweeps and matching code so that later watermark papers report results at a declared operating point.
References
- Generative Model Watermarking Based on Human Visual System
- Robust Watermarks Leak: Channel-Aware Feature Extraction Enables Adversarial Watermark Manipulation
- Machine vision-aware quality metrics for compressed image and video assessment
- Single Image Deraining: A Comprehensive Benchmark Analysis
- Machine-oriented Visual Media Quality Assessment
- Invisible Image Watermarks Are Provably Removable Using Generative AI
- Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline
- A Color Image Watermark Algorithm Based on Logistic and DWT-SVD
- Blind Robust Video Watermarking Based on Adaptive Region Selection and Channel Reference
- An optimized digital watermarking algorithm in wavelet domain based on differential evolution for color image
- Microsoft COCO: Common Objects in Context
- ReRAW: RGB-to-RAW Image Reconstruction via Stratified Sampling for Efficient Object Detection on the Edge