Per-Image Entropy Weighting of Masking Cues for Just-Noticeable-Difference Models

People are bad at seeing small distortions inside busy texture such as grass, gravel or fur, and good at seeing them on flat regions such as sky. This effect, called texture masking, is what lets watermarking, compression and quality metrics hide or discard signal where viewers won't notice. Models of it compute a just-noticeable-difference (JND) map, a per-pixel estimate of how much change a viewer can tolerate. They build this map from several cue maps (brightness adaptation, edge strength, orderly texture, disorderly texture, saliency) and then combine the cues. Today the combination is either a fixed hand-set formula or a large network trained on scarce human-labeled data.

We propose a middle path. The combination weights for each image are computed in closed form by the Entropy Weight Method. This technique comes from multi-criteria decision-making. It gives more weight to an indicator the more it varies across the items being ranked. Here the items are the patches of one image, and the indicators are the masking cues. The claim is simple. In each image, the cue that best separates "where noise hides" from "where it shows" is the one that varies most across that image. A few dispersion statistics per image should therefore recover most of the scene-to-scene variation that fixed formulas miss, with no training, and they stay readable as weights. The test is whether this per-image adaptivity beats a fixed formula, a globally fitted set of weights, and a trained deep JND model at matched perceptual quality.

Why this intersection, and what the obvious version gets wrong

The masking side supplies the cue maps. Visual Attention Guided Pixel-Wise Just Noticeable Difference Model splits an image into structure and texture using relative total variation. It then builds separate edge, orderly-texture and disorderly-texture masking maps plus saliency, and combines them with a fixed nonlinear additivity rule. That rule is our starting point. Its exponents and fusion constants are hand-calibrated and do not change from image to image. Two deep lines replace the rule with learning. Deep Learning of Human Visual Sensitivity in Image Quality Assessment Framework (DeepQA) learns a sensitivity map that multiplies the error map. It is trained from opinion scores and works on luminance only. MetaJND: A Meta-Learning Approach for Just Noticeable Difference Estimation learns cross-modal fusion weights with Swin Transformers over RGB, saliency and depth. Both motivate this proposal, because they show adaptive fusion matters. Neither exposes why a cue gets weight in a given image, and both need labeled data that MetaJND itself describes as expensive.

The weighting side supplies the mechanism. In the standard Entropy Weight Method pipeline, used for example in The Impact of Two-way FDI on GreenDevelopment Quality of the Logistics Industry– Based on the Threshold Effectof Environmental Regulation, each indicator is normalized across samples. Its Shannon entropy is computed, and the weight is proportional to one minus that entropy. The only prior vision use in the file is Cyclic Test Time Augmentation with Entropy Weight Method. It weights augmented views' softmax outputs at the decision stage and gains a modest 0.23 mean corruption error on ImageNet-C (73.57 to 73.34). That gain constrains our expectations: entropy weighting is a small-effect tool unless the thing being weighted truly varies in usefulness.

The obvious combination is to run the Entropy Weight Method once over a dataset to fix global weights for the masking cues. That version is weak, and a reviewer would say so. Weights set by dataset-wide dispersion have no link to human visibility. A five-parameter regression fitted to any JND dataset would beat them. Our version differs in two ways. First, the samples are patches within one image, so weights change per image. Second, the method's output is not trusted blindly. It is tested against exactly that fitted regression, and against a fitted version that also adapts per image.

The file flags an apparent contradiction (its Gap 4). The Entropy Weight Method rewards high dispersion, while masking says high-variance texture hides errors. The two do not conflict once the roles are separated. The entropy weight decides which cue map shapes the spatial allocation in this image. The cue map itself still says textured means a higher threshold. A disorderly-texture map that is high everywhere, as in an all-grass photo, has high entropy and low utility. It cannot tell the model where noise is safer, so it should cede influence to edges or brightness. This is our reading of how the two ideas relate. The weight-flip ablation below tests it directly.

Core Claim

For each image $I$ we compute masking cue maps $m_j$ (edge, orderly texture, disorderly texture, saliency, and a chroma-contrast cue we add to address the luminance-only limitation of DeepQA). We pool them over $n$ patches, rank-normalize them to $\tilde m_{ij}$ to blunt the outlier sensitivity the file notes for min-max scaling, and form

$$ p_{ij}=\frac{\tilde m_{ij}+\varepsilon}{\sum_{i}(\tilde m_{ij}+\varepsilon)},\qquad w_j^{(I)}=\frac{1-e_j}{\sum_k (1-e_k)},\quad e_j=-\frac{1}{\ln n}\sum_{i=1}^{n}p_{ij}\ln p_{ij}. $$

Here $p_{ij}$ is cue $j$'s share at patch $i$, $e_j\in[0,1]$ is its within-image entropy, and $\varepsilon$ is a fixed small constant that avoids the zero-log degeneracy. We use proper sum normalization rather than raw min-max values, so that $p_{ij}$ is a distribution. We fix $\varepsilon$ and the patch size $n$ once on a held-out split rather than per image, because small-sample instability is a known weakness of the method.

The JND map keeps the published model's brightness-adaptation term and overlap correction. It replaces the fixed masking combination with $\sum_j w_j^{(I)} m_j(x)$ scaled to the same range. The claim is that this training-free, per-image weighting beats the fixed rule. It should match or beat a globally fitted weight vector, and it should come within a small margin of MetaJND at a tiny fraction of the cost.

Research Program

The work has three parts. First, we build the pipeline on top of a reimplementation of the attention-guided JND model and report the per-image weight vectors. An analysis of how the weights shift across scene types (foliage, architecture, portraits, sky-dominated) is itself a deliverable, and it tests whether the weights are interpretable at all.

Second, we evaluate the JND maps with the standard noise-injection protocol. Noise shaped by the map is injected at a fixed PSNR, and perceptual quality is measured with SSIM and LPIPS plus a small subjective test. We also compare against ground-truth JND data.

Third, we test two downstream uses. The first is weighting the error map in full-reference quality assessment. The second is setting watermark strength in a localized embedder of the kind in Watermark Anything With Localized Messages. Its authors report visible repeated patterns in flat areas such as blue sky, even though its hand-crafted JND map raised PSNR from 24 to 36 dB. We ask specifically whether per-image reweighting shrinks strength in those areas without losing bit accuracy.

Baselines

The naive reference point is the fixed-rule model itself: Visual Attention Guided Pixel-Wise Just Noticeable Difference Model with its published constants. We also run an equal-weights variant. Both use identical cue maps, so any difference comes from weighting alone. That paper reports tolerating 0.86 dB more noise than Liu's model and 1.19 dB more than Wu's at equal SSIM on 12 standard 512×512 images. We rerun its protocol on the same images. Beating it means a further PSNR reduction at equal SSIM and equal or better subjective ratings.

The strongest cheap competitor is a globally fitted weight vector. This is ridge regression of the masking weights on PWJND training data, using the same cue maps and the same held-out split. It is also the most dangerous baseline, since it uses labels our method ignores.

We add a dispersion-conditioned fitted variant. It is a tiny model that maps the five per-image entropies $e_j$ to weights, trained on the same data. It separates whether the entropy signal matters from whether the closed-form mapping is the right use of it.

The strongest learned method is MetaJND: A Meta-Learning Approach for Just Noticeable Difference Estimation. It reports a 24.11% average bit-rate saving across JPEG, HEVC and VVC on MCL-JCI, and outperforms eight models on SSIM and LPIPS on KonJND-1k. The file gives no reported scalar for KonJND-1k or MDTJND, so we recompute MetaJND on those benchmarks from released weights under our protocol. We count "beating" as matching its bit-rate saving within two points on MCL-JCI, or exceeding it on cross-dataset MDTJND. Either result would have to come with orders of magnitude less compute, which we report in FLOPs alongside the 7.15-GFLOP pre-filter of JND-Guided Light-Weight Neural Pre-Filter for Perceptual Image Coding as a cost reference.

For quality assessment, the reference is DeepQA. It reports a weighted average Spearman rank correlation (SRCC) of 0.949 across LIVE, CSIQ, TID2008 and TID2013. We do not expect a training-free weighting to beat that overall. The target is instead the per-distortion SRCC on TID2013's chromatic distortion types, where DeepQA's luminance-only design is a documented weakness.

Two ablations isolate the key ingredient. The shuffle ablation uses entropy weights computed on a different random image. It keeps the weight distribution but destroys per-image correspondence, which tests adaptivity. The flip ablation uses weights proportional to $e_j$ instead of $1-e_j$, which tests the direction argument above.

Why not just…?

Why not just fit the masking weights once by regression on a JND dataset? This is the real threat. If a global fitted vector matches our method on KonJND-1k and MDTJND, per-image adaptivity adds nothing and the entropy machinery is decoration. The ridge baseline and the shuffle ablation settle it directly. If both tie our method, we report the negative result and fall back to the fitted weights as the recommendation. If the dispersion-conditioned fitted model beats the closed form, the finding becomes "within-image entropy is the right input feature," and we report that honestly.

Why not just train an end-to-end network such as DeepQA or MetaJND? Those models perform well. DeepQA reaches 0.949 SRCC. But they need scarce subjective labels, they hide why a cue matters, and DeepQA cannot see color artifacts. Our comparison accepts some loss in raw accuracy. It asks whether a zero-parameter method closes most of the gap from the fixed rule to MetaJND, and whether it generalizes better cross-dataset (MDTJND), where learned fusion is most prone to overfitting.

Why not just use local variance or gradient magnitude as the masking map? A single complexity measure cannot tell orderly repeating texture from chaotic texture. The attention-guided JND paper shows that this confusion overestimates masking in regular patterns. We include a local-variance JND as an extra reference in the noise-injection test. If it matches our method, the multi-cue structure is unnecessary for that benchmark.

Proposed Experiments

The decisive check is noise injection at fixed PSNR on the 12 standard images, KonJND-1k and MDTJND, scored by SSIM, LPIPS and a 20-assessor subjective test mirroring the original protocol. The method succeeds only if it beats the fixed rule and the shuffle ablation there, and at least ties the ridge baseline.

The coding check plugs each JND map into JPEG, HEVC and VVC preprocessing on MCL-JCI and reports bit-rate saving against MetaJND's 24.11%.

The quality-assessment check replaces DeepQA's learned sensitivity map with ours on TID2013 and reports per-distortion SRCC, focusing on chromatic types.

The watermarking check replaces the JND map in a Watermark-Anything-style embedder trained on COCO. It measures PSNR, bit accuracy on 10%-area splices (reported baseline 95.3%), and a flat-region artifact score: residual energy inside sky or flat-region masks. It succeeds if flat-region energy falls with bit accuracy within a point.

We will count the idea as failed if per-image weights do not beat their shuffled counterpart on the first check, whatever the other results show.