Most perceptual image-quality metrics first compute a map of how far the test image is from the reference at every location, then collapse that map to one number by averaging it. When a flaw covers a small area, averaging hides it: a mangled hand in an otherwise clean image contributes almost nothing to the mean. The obvious robust-statistics fix is to replace the mean with the median, and that fix is wrong. The median is built to ignore anything affecting less than half the data, so it would ignore small defects even more thoroughly than the mean does. The useful move from outlier detection is different. Use the median and the median absolute deviation (MAD, the median distance of each value from the median) of the error map to estimate what "ordinary" error looks like in this particular image. Then score the image separately on how much of the map exceeds that baseline, and by how much.
The claim is that perceptual quality is better predicted by two numbers than by one. The first is a robust baseline error level, which captures global degradation such as blur or noise. The second is an exceedance term measured relative to that baseline, which captures localized defects. The baseline cannot be inflated by the defects themselves, just as a fleet-median health baseline is not masked by one failing machine. This is a training-free change to the pooling stage of existing metrics such as LPIPS and GMSD. It is cheap, since a median over an H×W map costs almost nothing. Its predictions are falsifiable on restoration benchmarks with generative-model artifacts and on generated-image defect datasets that include pixel masks.
Core Claim
Let $d(p)$ be a metric's spatial distance map at location $p$ (for example, an LPIPS layer's channel-normalized squared difference before its spatial average, or GMSD's gradient-similarity deficit $1-\mathrm{GMS}(p)$). Define the baseline $m=\mathrm{median}_p\, d(p)$ and scale $s = 1.4826\cdot\mathrm{median}_p|d(p)-m|$, where the constant makes $s$ comparable to a standard deviation under normality. The exceedance field is
$$e(p) = \max\big(0,\; d(p) - m - k s\big),$$
where $k$ is a robust-z threshold. We summarize it by the fraction of locations with $e(p)>0$ and the mean of $e$ over those locations. The final score is a monotone combination of $m$ and these two exceedance statistics, with three or four parameters fit once on one dataset and then frozen for all others.
The specific prediction is about localized defects. Holding the global error level fixed, a defect covering $a\%$ of the image moves $m$ essentially not at all while $a<50$, and it moves the exceedance statistics sharply. Under mean pooling, by contrast, the same defect's influence shrinks linearly with $a$. Standard-deviation pooling responds to the defect too, but it mixes baseline spread with defect severity and is inflated quadratically by isolated single-pixel spikes. These predictions differ in ways a controlled experiment can separate.
Relation to Prior Work
Two papers motivate the gap. AGIDefect-4K builds a 4,000-image generated-image dataset with expert defect masks and quality scores, and reports that over 85% of defects cover less than 5% of the image, which is exactly the regime where averaging fails. PIPAL collects Elo-based human ratings on outputs of generative-model restoration and shows existing metrics degrade sharply there; its best existing metric, PieAPP, reaches only 0.71 SRCC.
The closest pooling work is GMSD. GMSD replaces mean pooling of a gradient-similarity map with its standard deviation, arguing that people notice uneven quality. We keep its insight that spread matters. We depart from it by separating the robust center from the tail rather than folding both into one second moment that has zero breakdown point.
Shift-Tolerant LPIPS fixes a different failure. By adding anti-aliasing BlurPool layers, it cuts LPIPS's 1-pixel rank-flip rate from 2.81% to 0.57%. It keeps spatial mean pooling, so it is complementary to our change, and we apply our pooling on top of it.
The method borrows from outlier detection. AQUAVS scores a sample by counting how many latent dimensions fall outside class median ± MAD. Our exceedance fraction is the spatial analogue: we count image locations rather than latent dimensions, and we use the image itself rather than a class as the reference population. Hendrickx et al. supply the "untainted baseline" logic. Medix uses element-wise medians to separate in-distribution from out-of-distribution samples, but it works on gradients across a dataset rather than locations within an image.
The one existing median-based quality method, DMS-IQA, takes the median over hundreds of noisy copies of the input to certify no-reference scores against adversarial attacks. It is roughly 300× slower, and it operates on the axis of noise samples, not spatial locations. It does not address defect dilution, and we do not claim adversarial certification. BiRQA likewise targets adversarial robustness with learned analytic features. Its clean accuracy of 0.988 SROCC on LIVE shows that classic benchmarks are nearly saturated, which constrains where we expect gains: on generative-model artifacts, not on classic distortions.
Research Program
We would build the pooling as a drop-in replacement for the spatial-average step. We apply it to the per-layer maps of LPIPS and Shift-Tolerant LPIPS and to the GMSD map, and we combine layers by the metric's existing learned layer weights so that no backbone is retrained.
Two engineering questions need answers first. The first is spatial scale: raw pixel maps contain isolated boundary spikes, so we compute exceedances on feature-layer maps whose receptive fields already aggregate a neighborhood. We add an optional minimum connected-area filter and ablate it. The second is the threshold $k$: we sweep it from 2 to 4 and report sensitivity rather than choose it on test data.
For the no-reference setting of generated images, where no reference exists, we apply the same pooling to the patch-wise quality map of an existing no-reference model. This tests whether the idea transfers beyond full-reference distance maps.
The analysis has three parts. The first is a controlled dose-response study. We paste localized distortions of known area and strength into KADID-10k reference images and chart how each pooling's score moves with area at fixed strength. The prediction is that exceedance pooling stays sensitive down to 1–5% area while mean pooling decays linearly. There are no human scores for these synthetic images, so this part tests only the mechanism, not whether it tracks perception.
The second part tests perceptual correlation on real human data: PIPAL, overall and on its generative-artifact subset; KADID-10k; LIVE; CSIQ; and TID2008 as a no-regression check.
The third part tests localization on AGIDefect-4K. We ask whether the exceedance field overlaps the expert masks (pixel AUC and IoU), and whether image-level exceedance separates defective from flawless images (detection AUC).
Baselines
The do-nothing reference is each backbone metric with its native mean pooling: standard LPIPS, Shift-Tolerant LPIPS, and GMSD's own map summarized by its mean. On the PIPAL generative-artifact subset, LPIPS reaches 0.4862 SRCC and LPIPS with BlurPool reaches 0.5135, both reported in PIPAL. Exceeding 0.5135 with the same backbone and only a pooling change is the first bar.
The strongest pooling competitor is GMSD's standard-deviation pooling, applied to the identical maps. It is the only published alternative to averaging that targets uneven quality, and it costs the same as our method. There is no reported result on Lacuna for GMSD's SRCC on these benchmarks, so we recompute it ourselves on the same maps.
We also run simple tail pooling, meaning a high quantile (the 90th, 95th, or 99th percentile) and Minkowski pooling with an exponent above 2. These are what a reviewer would try first. They get the same few-parameter fit, on the same fitting split, with the same grid-search budget as our method.
The strongest full-benchmark reference is PieAPP at 0.71 SRCC on PIPAL (PIPAL). For generated images, the reference is the trained AGIDA model: 0.65 detection AUC and 0.800 quality SRCC on AGIDefect-4K (AGIDefect-4K). AGIDA is supervised on that dataset's labels, so matching it with a training-free pooling change would be a strong result. Beating it is not required.
The key ablation replaces the median-referenced exceedance with a raw high-quantile exceedance, $\max(0, d(p)-q_{0.9})$, which removes the robust baseline while keeping tail sensitivity. A second ablation replaces the median and MAD with the mean and standard deviation inside the same exceedance formula. That swap isolates the high-breakdown ingredient, and the mechanism is contradicted if it matches the full method.
Why not just…?
Why not just use median pooling instead of mean pooling? Because the median is designed to ignore any feature that affects under half the locations, it discards exactly the small defects we care about. Our dose-response study includes plain median pooling and should show it flat for defect areas below 50%. That result is the reason the proposal uses the median as a reference level rather than as the summary.
Why not just pool with a high percentile or max? This is the real threat. A 95th-percentile pool is tail-sensitive, trivial to compute, and may capture much of the gain. Where it should fail is in mixing two things. An image with uniformly heavy noise and an image that is clean except for one bad region can have the same 95th percentile yet look very different to people. Median-referencing separates those cases. The raw-quantile ablation settles the question directly: if it matches our method on the PIPAL generative subset and on AGIDefect-4K localization, we would report that the benefit comes from tail pooling alone and that the robust baseline adds nothing.
Why not just use GMSD's standard deviation? GMSD already shows that spread beats the mean, but its standard deviation is a single number that grows both with benign baseline variation and quadratically with isolated spikes. We compare against it on identical maps. We also inject single-pixel boundary artifacts to test the specific prediction that its score is inflated by spikes that humans ignore.
Why not just train a learned attention pooling or a preference model? Learned models such as AGIDA or HPS fit their training distribution but carry known biases. HPS reaches only 43.5% choice accuracy, and zero-shot multimodal models score 0.29–0.38 defect-detection AUC on AGIDefect-4K. Our contribution is a training-free, interpretable pooling that transfers across datasets. We report it frozen across datasets against supervised models, and we concede it if trained pooling wins by a wide margin under cross-dataset transfer.
Decisive Experiments
The proposal succeeds if three things hold. First, exceedance pooling on Shift-Tolerant LPIPS exceeds 0.5135 SRCC on the PIPAL generative subset and beats both standard-deviation pooling and raw-quantile pooling. Second, it holds within 0.01 of native pooling on LIVE and CSIQ. Third, its exceedance field localizes AGIDefect-4K masks better than the mean-pooled map's own peak locations.
The proposal fails, and should be reported as failing, in either of two cases. One is if the raw-quantile ablation matches it. The other is if the mean-and-standard-deviation variant matches it, which would mean the high-breakdown property is not what does the work.
References
- AGIDefect-4K: A Richly Annotated Dataset for AI-Generated Image Defect Detection, Localization and Explanation
- PIPAL: A Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration
- Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index
- Shift-Tolerant Perceptual Similarity Metric
- Assessing the quality of the datasets by identifying mislabeled samples
- Similarity-based anomaly score for fleet-based condition monitoring
- A Median Perspective on Unlabeled Data for Out-of-Distribution Detection
- Stochastic BIQA: Median randomized smoothing for certified blind image quality assessment
- BiRQA: Bidirectional Robust Quality Assessment for Images
- Human Preference Score: Better Aligning Text-to-image Models with Human Preference