Super-resolution networks, which turn a small image into a larger and sharper one, often mistake sensor grain and compression debris for real detail. They then sharpen that debris into crisp, crystalline patterns. This failure is called artifact magnification, and it is worst on real photographs, whose noise looks nothing like the clean, synthetically shrunk images most models are trained on. On real photos there is usually no clean high-resolution reference to compare against, so nobody can say which regions of an output are trustworthy. This proposal borrows a tool from astronomy to fill that gap. Astronomers decide whether a star truly varies by looking at many repeated observations. They ask whether its brightness spreads more than a nearby steady comparison star's does, measuring the spread with outlier-resistant statistics, and they ask whether the fluctuations are smooth and structured or just jitter. We apply the same logic to a super-resolution model. We feed it several versions of the same input that differ only in their noise, and per region we measure how much the output moves, relative to how much a plain bicubic upscale moves. Real texture belongs to the scene and stays put across these versions. Magnified noise follows the noise and moves with it.
The result is a reference-free map of artifact magnification: a quality estimate that never sees the ground truth. Because it is computed from the model's own response to its input, rather than learned from a training set of synthetic degradations, we expect it to survive the jump from synthetic to real data better than learned quality predictors do. That transfer claim is what we test. If the map works, it also gives a principled way to decide region by region where to trust a sharp generative upscaler and where to fall back to a conservative one.
Core Claim
Artifact magnification can be posed as a variability-detection problem. The thing that "varies" is the model output across noise realizations of a fixed scene, and the steady comparison source is a linear upsampler run on the same realizations. Two robust statistics, borrowed directly from time-domain astronomy, separate the cases we care about. The first is an amplitude ratio built from interquartile ranges, which says whether the model amplifies input noise beyond simple pass-through. The second is an inverse von Neumann ratio computed on the deviation field, which says whether the amplified component is spatially structured, as crystalline or checkerboard artifacts are, or white, as harmless grain is. We claim three things. Together the two statistics predict where full-reference error exceeds the bicubic floor on real benchmarks. They do so better than learned no-reference quality models, input noise-level maps, and sampling-based uncertainty given the same compute. And their predictive power drops much less than learned predictors' when calibration is done on synthetic degradations and testing on real cameras.
Where this sits in prior work
The failure itself is well documented. Unsupervised Learning for Real-World Super-Resolution showed that bicubically trained models magnify smartphone sensor noise. They closed the gap by learning a realistic degradation generator and training on synthetic pairs, but they validated only with reference metrics. Super-resolving Noisy Images described the deadlock in which denoising first erases texture while skipping denoising amplifies noise. Their answer was to blend a clean candidate with a noisy candidate, guided by local pixel variance and a kurtosis prior. That work evaluated only synthetic Gaussian noise and admits failure on regular structures such as window grids. Our method keeps their two-candidate idea and their use of local dispersion, but changes what is dispersed. They measure spread across pixels of one image, which confuses texture with noise by construction. We measure spread across noise realizations of the same pixel, which does not. Not All Degradations Are Equal gates features with a learned Fourier-domain noise detector and reports gains up to 1.50 dB PSNR on the Canon DSLR set. Their detector is an internal gate trained on synthetic noise, not a calibrated quality estimate. Whether a learned detector transfers is precisely what we contest, so their model serves both as a restorer we evaluate and as a source of a competing detector. Towards Realistic Data Generation for Real-World Super-Resolution learns real degradations with a diffusion model and filters candidates by latent consistency. It also improves training data, which is complementary to measuring output trustworthiness.
On the quality-estimation side, the speech literature motivates the reference-free goal and constrains our design. MetricGAN-U trains an enhancer against learned non-intrusive metrics (DNSMOS, SRMR) without clean data, and the research file notes that such loops tend to game the metric's blind spots. Clean-prior approaches score degraded inputs by likelihood under a model of clean signals (Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speech) or by quantization error (Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech). Those methods measure how far the signal is from clean, not whether a processing step made it worse, and the latter is the question for magnification. Given the gaming risk, we use our statistic for evaluation and region selection, not as a training loss.
The statistical method comes from astronomy. The Hubble Catalog of Variables pairs the interquartile range, which captures amplitude while ignoring outliers, with the inverse von Neumann ratio, which captures whether successive values are correlated. It uses the two together to separate real variables from detector noise. Zhou et al. compare a target's dispersion with a simultaneously observed steady star, so that a ratio near one means "only the environment varies." Fractional variability, as used by Aleksić et al. and MOMO – V, subtracts the expected noise variance from the observed variance. The astronomy work also carries a warning. In A search for radio emission from double-neutron star merger GW190425 using Apertif, all 55 single-epoch transient candidates were imaging artifacts, and calibration drift caused 9–14% inter-epoch flux scatter. Thresholding raw variability without a comparison reference produces confident false detections. That is why our statistic is always a ratio against a reference processed identically.
Research Program
Let $x$ be a real low-resolution input, $S$ the super-resolution model under test, and $U$ bicubic upsampling. We form $K$ realizations $x_k$ of the same scene in one of two ways. In single-image mode, we add small independent perturbations whose local level matches an estimate of the input's own noise. In real-epoch mode, we use flow-aligned neighbouring video frames, which are genuinely repeated observations. For each patch $p$ we compute $A_p$, the median over pixels of the interquartile range of $S(x_k)$ across $k$, and $B_p$, the same quantity for $U(x_k)$. The amplification ratio is
$$C_p = A_p / B_p .$$
A value near one means the model merely passes the noise through. Values well above one mean it amplifies the noise. Because $U$ sees identical realizations, misalignment in the video mode inflates $A_p$ and $B_p$ together and largely cancels, which mirrors how a comparison star absorbs calibration drift. We do not use the square-root fractional-variability form, whose argument goes negative at low signal (a known degeneracy). Instead we report the excess $\max(0, A_p^2 - B_p^2)$ alongside the ratio, and we record how often the clip activates. For structure, let $d_k = S(x_k) - \mathrm{median}_j\, S(x_j)$ be the deviation field. We compute the inverse von Neumann ratio $1/\eta_p$ of $d_k$ along rows and columns of the patch, then take the median over $k$. The original statistic orders observations in time; here it serves as a lag-one smoothness measure in space, so high values flag coherent stripes, grids, and crystalline patterns rather than white grain.
The program has four parts. First, we build the estimator for convolutional, GAN-trained, and one-step diffusion super-resolution models. Second, we test whether the single-image perturbation mode agrees with the real-epoch mode. This checks the linearization assumption that a model's sensitivity to added noise reflects its sensitivity to the noise already present. Third, we calibrate a simple two-feature decision rule (thresholds or a logistic fit on $C_p$ and $1/\eta_p$) on synthetic data and test it on real cameras. Fourth, we use the map to choose, per region, between a sharp output and a conservative one, which extends the two-candidate blending of Singh et al. with a criterion that does not mistake stochastic texture for noise.
Baselines
The naive reference point is to do nothing. Each model's output is trusted everywhere, and the artifact "map" is constant. For the selection experiment this means using the sharp model alone, or the conservative model alone, over the whole image.
The first competitor is an input noise-level map: a standard local noise estimate computed on $x$, with high noise taken to predict artifacts. It costs one pass and asks whether looking at the model's response adds anything beyond looking at the input. Our map beats it if the partial correlation with patch error stays clearly positive after controlling for input noise.
The second competitor is a learned no-reference quality model. MUSIQ is the natural choice because One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation reports a MUSIQ score of 65.86 for RSD on RealSR, versus 59.87 for its teacher ResShift. We will also train a small patch-level error predictor of matched capacity on DIV2K with synthetic degradations, which is the learned detector most directly threatened by the synthetic-to-real gap. The learned detector from Not All Degradations Are Equal is a third learned option. All learned predictors get the same training images and the same hyperparameter search budget as the calibration of our two-feature rule.
The third competitor, and the most dangerous, is sampling-based uncertainty given the same number of forward passes $K$. For stochastic generators this is the spread across diffusion samples; for deterministic networks it is spread across dropout masks or a small ensemble. We will also give this baseline the same interquartile-range statistics, so that the comparison isolates what is perturbed (the input noise, against a reference) rather than how spread is summarized.
Three ablations isolate the key ingredients. Replacing interquartile range with standard deviation tests whether robustness matters. Dropping the bicubic reference, using raw $A_p$ alone, tests the comparison-star idea. Dropping $1/\eta_p$ tests whether structure adds information beyond amplitude.
For the estimation task, no reported result found on Lacuna for reference-free prediction of per-patch super-resolution error on RealSR or DRealSR, so every number there will come from our own runs under the shared protocol. For the selection task, the reference numbers are the 1.50 dB Canon gain reported for TFD and RSD's MUSIQ of 65.86 on RealSR. Beating the baseline means lowering LPIPS on RealSR relative to the better of the two candidates, without dropping MUSIQ below the sharp candidate's own score under our evaluation.
Why not just…?
Why not just use an existing no-reference quality model such as MUSIQ? Such models are learned from human ratings that partly reward sharpness. The research file documents, through the MetricGAN-U line, that learned non-intrusive metrics have blind spots that high-frequency patterns can exploit, and magnified artifacts are high-frequency patterns. A metric that rewards crispness may score crystalline noise as detail. This is an empirical question, so the estimation experiment reports MUSIQ's patch-level correlation with excess reference error directly. If MUSIQ matches our map on real cameras, our method's case reduces to cheapness and interpretability, and we will say so.
Why not just use sampling or ensemble uncertainty? This is the real threat. Diffusion-sample spread measures generative ambiguity, meaning how many sharp images fit the input, which is high on clean but ambiguous regions such as the distant foliage RSD's authors flag. It need not be high where noise is being amplified. Dropout spread on a deterministic network has no tie to the input noise at all. Our ablation that keeps the robust statistics but swaps the perturbation source settles the matter at equal $K$. If sampling spread wins on RealSR, the astronomy framing contributes only the reference ratio, and we would refocus on adding that ratio to sampling spread.
Why not just estimate the input noise level and denoise where it is high? High input noise does not imply amplification: a model trained on realistic degradations may suppress noise that a bicubic-trained model sharpens. Singh et al.'s deadlock shows that denoising those regions also destroys texture. Their sequential denoise-then-upscale baseline reached 25.05 dB, against 25.31 dB for blending, on the Lama image at σ=20. The partial-correlation test against the noise-level map and the selection experiment, where noise-gated fallback is one arm, decide this question.
Why not just use standard deviation? Hallucinations appear in a few realizations and act as outliers, and the Hubble catalogue work shows squared-deviation statistics are dominated by such outliers. Even so, standard deviation might suffice when $K$ is small, and the ablation measures it.
Proposed Experiments
The estimation experiment uses RealSR and DRealSR, which provide real high-resolution references only for scoring. The models under test are an ESRGAN-style network trained on bicubic data, the same backbone trained on Lugmayr-style realistic pairs, TFD on SRResNet, and RSD. The target per patch is excess error, defined as the patch LPIPS of $S(x)$ minus the patch LPIPS of $U(x)$, which isolates damage the model added. We report Spearman correlation and area under the precision-recall curve for flagging the worst-decile patches, at $K \in \{4, 8, 16\}$.
The transfer experiment calibrates every method on DIV2K with synthetic noise, blur, and JPEG degradations. It then tests without retuning on RealSR, DRealSR, the Canon and Nikon DSLR sets, and the 891-pair SmartPhone set used by Peng et al. The decisive quantity is the drop in correlation from synthetic to real. We predict the drop is substantially smaller for our map than for the trained patch predictor. If the drops are similar, the central claim fails.
The consistency experiment compares single-image and real-epoch maps on RealVSR frames using rank agreement. No ground truth is needed, which directly tests whether synthetic perturbations can stand in for real repeated observations.
The selection experiment fuses RSD or ESRGAN outputs with a conservative L1-trained output per patch. The weight comes from our map; the arms are global choice, noise-gated fallback, Singh-style pixel-variance blending, and sampling-spread gating. We score LPIPS and PSNR on RealSR and DRealSR and MUSIQ as a secondary measure, and we separately report results on Urban100-style regular structures, where Singh et al.'s method is known to fail.
Two results would most change our minds. The first is sampling spread matching our map at equal $K$. The second is the single-image and real-epoch modes disagreeing, which would confine the method to video and burst capture.
References
- Unsupervised Learning for Real-World Super-Resolution
- Super-resolving Noisy Images
- Not All Degradations Are Equal: A Targeted Feature Denoising Framework for Generalizable Image Super-Resolution
- Towards Realistic Data Generation for Real-World Super-Resolution
- MetricGAN-U: Unsupervised Speech Enhancement/ Dereverberation Based Only on Noisy/ Reverberated Speech
- Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speech
- Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
- The Hubble Catalog of Variables
- OPTICAL VARIABILITY OF THE RADIO SOURCE J 1128+5925
- The 2009 multiwavelength campaign on Mrk 421: Variability and correlation studies
- MOMO – V. Effelsberg, Swift, and Fermi study of the blazar and supermassive binary black hole candidate OJ 287 in a period of high activity
- A search for radio emission from double-neutron star merger GW190425 using Apertif
- One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation