Image denoisers are trained and ranked by how close their output is to a clean photograph, pixel by pixel. That measure treats every unit of leftover error the same, whether it sits in fine, grain-like speckle or in broad, smooth color blotches that span dozens of pixels. People do not treat them the same. A faint, low-contrast blotch on a flat wall looks like a stain. The same amount of error spread as fine grain over a textured surface is close to invisible. This proposal asks how much human annoyance each spatial scale of residual error causes, measures it directly on the outputs of modern denoisers, and then trains denoisers to move their unavoidable error budget out of the scales people notice and into the scales they ignore.
The idea in two sentences: decompose a denoiser's residual error (output minus ground truth) into a pyramid of spatial-frequency bands, and run a human study to learn how strongly each band, in flat versus textured regions, drives perceived quality. Then use those measured weights as a training loss. The prediction is a small loss in PSNR (peak signal-to-noise ratio, the standard log-scaled pixel error) and a clear gain in human preference. The gain should come from removing low-frequency blotches that both PSNR and texture-oriented deep metrics underweight.
Why this gap is real
Two literatures meet here without having been joined. Work on hierarchical denoising, meaning methods that process an image at several downsampled resolutions, exists because local filters cannot see large artifacts. NL-Bayes, a patch-based denoiser that pools similar small patches across the image, explicitly concedes that inside a low-frequency blotch every small patch looks smooth and self-similar, so the blotch survives. Multi-Scale DCT Denoising builds a pyramid specifically to remove these blotches in smooth areas. These papers motivate the proposal by establishing that coarse-scale residuals are a distinct and stubborn failure mode. Modern deep hierarchical denoisers such as MADNet, MSFN, NBNet and MDRN build multi-scale processing into their architectures. They then train with uniform pixel losses (Charbonnier, L1) and report only PSNR and SSIM, so nothing in their objective says which scale's error matters to a viewer.
On the evaluation side, several papers document the gap between metrics and human judgment, and they constrain what this proposal can trust. PIPAL shows PSNR and SSIM become anti-correlated with humans on realistic restorations. Human Preference Score argues deep feature extractors favor local texture over global structure. That is exactly the bias that would make a metric like LPIPS (a learned similarity built on ImageNet network features) insensitive to smooth, large blotches. Shift-Tolerant Perceptual Similarity Metric shows LPIPS flips rankings under one-pixel shifts. PAMSE supplies part of the method: blurring both images with a Gaussian before computing MSE approximates the eye's contrast sensitivity and lifts TID2008 Spearman correlation from 0.7718 to 0.9162. It uses a single fixed blur, was never tested as a training loss for deep networks, and was never tested on denoiser residuals. Low-pass filtering implicitly upweights coarse error relative to fine error, which is the direction this proposal hypothesizes, but in uncalibrated form.
The closest existing work is Re²IC's Wavelet-Wasserstein Distortion. It splits network features into wavelet subbands and enforces strict fidelity on the low band while allowing distribution matching on high bands, inside per-image implicit compression. Its scale weighting is designed rather than measured, it operates in VGG feature space with one wavelet level, and it targets compression. This proposal differs in three ways. It measures human sensitivity per scale of the residual on real denoiser outputs. It conditions that sensitivity on local content. It tests whether a pixel-space loss built from the measurements beats designed alternatives in feed-forward denoising.
Core Claim
The claim has two parts. First, for real-noise denoisers trained on smartphone data, human-perceived quality depends on residual error energy in a strongly scale- and content-dependent way. Coarse-band error in flat regions is weighted far more heavily than PSNR implies, and more heavily than LPIPS-family metrics capture. Second, a denoiser trained with a loss weighted by these measured sensitivities will be preferred by humans over the same network trained with L1, LPIPS-augmented, PAMSE, or standard contrast-sensitivity-weighted losses, at a PSNR cost no larger than a few tenths of a dB.
Formally, let $x$ be the clean image, $\hat{x}$ the denoised output, $B_k$ the operator extracting band $k$ of a Laplacian pyramid (band 0 finest), and $m(x)$ a local activity map computed from $x$ that separates flat from textured regions. The proposed loss is
$$ \mathcal{L}(\hat{x}, x) = \sum_{k=0}^{K} \left\| \sqrt{w_k^{\text{flat}}\,(1-m_k) + w_k^{\text{tex}}\,m_k} \;\odot\; B_k(\hat{x}-x) \right\|_2^2 , $$
where $m_k$ is $m(x)$ resampled to band $k$'s resolution and $w_k^{\text{flat}}, w_k^{\text{tex}}$ are nonnegative per-band weights. The loss remains a weighted quadratic in $\hat{x}$, so it keeps the convexity and stability of MSE that PAMSE emphasized. The weights are not hand-set. They are fit to human paired comparisons with a Bradley–Terry model, which converts pairwise preferences into scalar scores. The probability that a viewer prefers stimulus $a$ over $b$ is modeled as $\sigma(\mathcal{L}_b - \mathcal{L}_a)$, with $\sigma$ the logistic function.
Research Program
The first stage builds a controlled stimulus set. We take NBNet, HINet and MADNet-style models trained with L1 on SIDD, run them on held-out SIDD and DND crops, and decompose each residual $\hat{x}-x$ into five pyramid bands in an opponent color space, so that luminance and chroma blotches are separated. This depth respects the roughly five-level limit beyond which HDCWNet found coarse decomposition destroys geometry. We then synthesize stimuli by adding to $x$ rescaled versions of the real residual bands. Pairs are matched in total MSE, and therefore in PSNR, but differ in how that energy is distributed across bands and across flat versus textured regions. Using real residuals rather than synthetic noise is what makes the measurement about denoisers rather than about generic distortion. Matching MSE is what makes the study informative, because PSNR is indifferent between every pair by construction.
The second stage collects two-alternative forced-choice judgments on these pairs, together with pairs of actual outputs from different denoisers and losses. Each comparison is shown with the reference. Viewing distance is fixed so that pyramid bands map to known cycles per degree. We fit the Bradley–Terry weights and report the resulting sensitivity profile with bootstrap intervals. We also compute each existing metric's implied profile, meaning its response to unit energy added in each band and region type, so that mismatches are visible directly rather than inferred from correlation coefficients.
The third stage retrains fixed backbones with the fitted loss. A light L1 term is added at the finest band only, to prevent grain from growing without bound. We then evaluate on held-out images and with fresh human raters who did not contribute to the weight fitting.
Baselines
The naive reference point is each backbone (NBNet and HINet, both hierarchical U-Nets with public SIDD results) trained with its published pixel loss. On SIDD, NBNet reports 39.75 dB PSNR (NBNet), MADNet reports 39.78 dB and SSIM 0.959 (MADNet), and MSFN reports 40.12 dB (MSFN), the highest PSNR in the file. We will include MSFN's released-style configuration as an untouched reference for what pure fidelity optimization achieves. Our retrained L1 baselines must reproduce their papers' PSNR within about 0.1 dB before any comparison is trusted. Beating this reference means a statistically significant human preference under matched training budgets, together with lower coarse-band flat-region residual energy.
The strongest perceptual competitor is L1 plus a deep feature loss, in two forms. The first is the VGG feature loss of Perceptual Losses for Real-Time Style Transfer and Super-Resolution. The second is the anti-aliased shift-tolerant LPIPS of Shift-Tolerant Perceptual Similarity Metric, which removes LPIPS's known one-pixel fragility (rank-flip rate down from 2.81% to 0.57% on BAPPS). Each receives the same training schedule and the same hyperparameter search budget over its mixing weight as our method receives over its regularizer. Selection is done on a validation human-judgment split, not on PSNR. A designed scale-decoupled loss in the spirit of WA-WD is reimplemented in pixel space for denoising, with strict L2 on the lowest wavelet band and a relaxed penalty on high bands. It serves as the direct test of whether measured weights beat designed ones. We know of no reported denoising result for WA-WD (no reported result found on Lacuna for WA-WD as a denoising loss); its reported 92.3% Spearman is for metric correlation on CSIQ, TID2013 and LIVE.
Two baselines target the ingredient being claimed. PAMSE used as a loss, with its blur width tuned on the same validation split, tests whether a single low-pass suffices. A Laplacian-pyramid loss weighted by a standard textbook contrast sensitivity curve, with no content dependence, tests whether human measurement adds anything beyond known psychophysics. A further ablation fits only flat-region-independent weights, removing $m(x)$ and isolating the contribution of content conditioning.
Why not just…?
Why not just add LPIPS or a VGG feature loss? That is the default fix for over-smoothing. The concern is a mismatch in mechanism: those features are tuned to texture (Human Preference Score), and the target artifact is smooth and large. The stage-two implied-profile analysis tests this directly. If shift-tolerant LPIPS turns out to respond strongly to coarse flat-region energy, and the LPIPS-trained network matches ours in human preference, then the hypothesis that deep metrics are blind to blotches is wrong. We would report that outcome as such.
Why not just use PAMSE, MS-SSIM, or a textbook contrast-sensitivity weighting? This is the most serious threat. A fixed contrast sensitivity curve already peaks at low-to-mid frequencies and may encode most of what we measure. The proposal survives only if the human-fitted, content-conditioned weights beat the tuned PAMSE loss and the CSF-weighted pyramid loss in held-out preference. That would happen most plausibly through masking: blotches matter on flat walls and sky but not on foliage. If the CSF baseline ties, the contribution shrinks to a measurement result. That result would still be useful, since it would show classical psychophysics is the right and nearly free loss for denoisers, but the method claim would be withdrawn.
Why not just use a bigger network, or non-local patch matching, to remove blotches? Capacity and context change how much total error remains. They do not change how a fixed loss distributes that error across scales. NL-Bayes's own analysis says patch similarity cannot see inside blotches (NL-Bayes), and MSFN at 40.12 dB still optimizes a uniform pixel loss. Including MSFN and two backbones of different size lets us check whether the preference gain persists as PSNR rises. If the strongest backbone's L1 output already shows negligible coarse-band flat-region residual, the problem disappears with scale and we would say so.
Proposed Experiments
Training uses SIDD; evaluation uses held-out SIDD crops and DND for real noise. CBSD68 and Urban100 with spatially correlated synthetic noise serve as controlled stress tests, since Urban100's flat architectural surfaces next to repetitive texture are where masking should matter most. We report PSNR and SSIM, LPIPS and shift-tolerant LPIPS, PAMSE, and a coarse-band flat-region residual energy diagnostic. The primary outcome is human preference from new raters, measured by paired comparisons converted to Bradley–Terry scores with confidence intervals. As a secondary external check we score outputs with BiRQA, a multiscale full-reference quality metric reporting 0.988 SROCC on LIVE, and with metrics validated on PIPAL's denoising subset. These are checks, not targets, and we never select hyperparameters on them.
Three results decide the question. First, the fitted sensitivity profile must differ reliably from PSNR's flat profile and from at least the LPIPS-family profiles in the coarse flat-region bands; otherwise the premise fails. Second, the measured-weight loss must win held-out human preference against the tuned CSF-pyramid and PAMSE losses, which isolates whether measurement and content conditioning add value. Third, the PSNR cost relative to the L1-trained same backbone must stay within roughly 0.3 dB while the coarse-band flat-region residual drops. That combination would show the error was moved rather than merely increased. A final transfer check refits weights on one backbone's residuals and applies them to the other's. If the profile is a property of human vision and image content rather than of one network's failure pattern, the gain should survive the swap.
References
- Implementation of the "Non-Local Bayes" (NL-Bayes) Image Denoising Algorithm
- Multi-Scale DCT Denoising
- Learning multi-scale spatial-frequency features for image denoising
- Multi-Scale Selective Feedback Network with Dual Loss for Real Image Denoising
- NBNet: Noise Basis Learning for Image Denoising With Subspace Projection
- Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation
- PIPAL: A Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration
- Human Preference Score: Better Aligning Text-to-image Models with Human Preference
- Shift-Tolerant Perceptual Similarity Metric
- Perceptual Fidelity Aware Mean Squared Error
- Frequency-Aware Perceptual Optimization for Low-Complexity Implicit Image Compression
- ALL Snow Removed: Single Image Desnowing Algorithm Using Hierarchical Dual-tree Complex Wavelet Representation and Contradict Channel Loss
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution
- BiRQA: Bidirectional Robust Quality Assessment for Images