Model-Referred Chromatic Quality for Retinal Systemic-Risk Predictors

Deep networks can now read signs of brain and heart disease from ordinary color photographs of the back of the eye. Examples include silent strokes seen on MRI, future stroke, and "retinal age." Yet the images these networks consume are uncalibrated. Flash strength, lens yellowing from cataract, fundus pigmentation and camera optics all shift their colors. The tools built to judge color image quality, such as perceptual metrics with explicit models of human color vision, answer a different question: will a person notice the difference? This proposal asks what quality means when the observer is the risk model rather than a human. We will measure, channel by channel and scale by scale, how strongly a retinal foundation model's outputs move under color and brightness perturbations that leave the anatomy unchanged. We will then turn that measurement into a quality score that predicts how far an image's risk estimate should be trusted.

The idea in two sentences is this. Human perceptual metrics weight distortions by how visible they are to people, but a systemic-risk model may be fragile in exactly the places people are not, such as slow, low-contrast shifts in the blue–yellow color axis across the background tissue. We therefore build a "contrast sensitivity function" for the model itself, test whether it predicts prediction drift better than human-referred metrics do, and check whether it explains real-world failures on low-quality images.

Core Claim

Our hypothesis is that systemic predictors built on fundus photographs carry a sensitivity profile that differs measurably from human color vision. We expect them to be more sensitive than humans to low-spatial-frequency chromatic drift in the parenchyma (the background retinal and choroidal tissue) and less sensitive to fine luminance noise. Because of this mismatch, display-referred human metrics will be poor predictors of how much a degradation corrupts a risk score, and a metric fitted to the model's own sensitivity will do better. The claim is new because every chrominance-aware quality model in the file is fitted to human opinion scores, and every retinal quality model in the file is fitted to human gradability labels. None uses the downstream model's output stability as the target.

The motivating evidence comes from both sides. Effects of Hypertension, Diabetes, and Smoking on Age and Sex Prediction from Retinal Fundus Images showed that a ResNet still predicts sex with AUC above 0.96 after vessels are inpainted away. That result means much of the systemic signal lives in parenchymal texture and color, exactly the region where acquisition-driven color shifts also live. Evaluation of Retinal Image Quality Assessment Networks in Different Color-Spaces showed that images humans call merely "Usable" already degrade automated diabetic-retinopathy grading. It did so with a classifier that fuses separate RGB, HSV and LAB backbones and is trained on human three-level grades. On the quality side, ColorVideoVDP: A visual difference predictor for image, video and display distortions supplies the method we reuse. Its castleCSF models human sensitivity separately for the luminance, red–green and blue–yellow channels, as a function of spatial frequency and background luminance, inside a differentiable Laplacian pyramid. We keep that decomposition and replace the human sensitivity curves with curves measured on the model.

Research Program

The first component is a perturbation audit. Starting from EyeQ images graded "Good," we apply perturbations that change acquisition but not anatomy. One family is band-limited contrast patterns injected into a single opponent channel at a single pyramid level. A second family mimics physical failures. It uses the illumination, haloing and defocus model from Modeling and Enhancing Low-Quality Retinal Fundus Images, extended with a blue-attenuating filter for lens brunescence, chroma blur, and global white-point shifts. Because the anatomy is fixed by construction, any change in a model's output is error, and this gives us ground truth without a calibrated camera.

We probe three kinds of downstream readout. The first is a RETFound-initialized predictor of systemic targets (A foundation model for generalizable disease detection from retinal images pretrains a ViT with masked autoencoding on 1.6 million retinal images). DeepRETStroke shares this architecture family, but we do not assume we can access it. The second is a disease grader trained on EyePACS labels. The third is a vessel-width readout: mean width along fixed centerlines from a DRIVE-trained segmenter. This readout stands in for CRAE/CRVE-style caliber measurement.

For each opponent channel $c$ and pyramid band $f$, we estimate the model's threshold contrast $T_m(c,f)$. This is the smallest perturbation contrast that moves the output by a clinically chosen tolerance $\delta$, for example a fixed change in predicted probability or one pixel-equivalent of width. We then compare $T_m$ with castleCSF's human threshold $T_h(c,f)$ at matched assumed display conditions.

The second component turns sensitivity into a metric. Given a reference image $x$ and a degraded image $x'$, we define the predicted drift

$$D_m(x,x') = \sum_{c,f} w_{c,f}\, \lVert P_{c,f}(x) - P_{c,f}(x') \rVert_2 ,$$

where $P_{c,f}$ extracts opponent channel $c$ at pyramid band $f$, and $w_{c,f}$ are nonnegative weights fitted to the observed output change $|y(x)-y(x')|$. Taking $w_{c,f} \propto 1/T_h(c,f)$ gives a human-referred analogue. Comparing the fitted $w$ with that analogue is the scientific test of the core claim.

Fundus images have no physical units, so we will repeat every comparison over a grid of assumed display peak brightness and black level. This tests whether the conclusions depend on the display-referred assumptions that ColorVideoVDP requires.

The third component carries the result to real images, which have no clean reference. We distill $D_m$ into a no-reference regressor that predicts expected drift from a single image. We then ask whether its scores track real downstream error on EyeQ quality strata better than human-grade classifiers do.

A fourth, secondary component uses the fitted sensitivity map to choose fine-tuning augmentations. The idea is to augment only along channel–band directions that are pure acquisition nuisance in the audit, not with generic color jitter. This tests whether the map is actionable rather than only descriptive.

Baselines

The naive reference point is to do nothing: run the predictor on every image with no quality score at all. For metric comparisons, the naive reference is luminance-only PSNR and SSIM, as described in A Hitchhiker's Guide to Structural Similarity. These are what most retinal enhancement work, including cofe-Net's reported PSNR 21.24 and SSIM 0.758, uses to judge fidelity. Beating them on the drift-prediction task is a low bar, and failing to beat them would end the project.

The strongest human-referred competitors are ColorVideoVDP and 3C-FUNQUE+. A FUNQUE Approach to the Quality Assessment of Compressed HDR Videos reports that 3C-FUNQUE+ computes structural similarity, detail loss and temporal features jointly across color channels in a shared wavelet space, with SROCC 0.8985 on LIVE-HDR under dark ambient conditions versus 0.8123 for VMAF. We also include the color-difference metric from Image Quality Assessment and Color Difference, which combines CIEDE2000 with color-naming shifts. These are the right competitors because they already encode chromatic sensitivity explicitly, so they directly test whether human color vision is a good enough proxy for model sensitivity. All of them are run on identical reference–degraded pairs, at identical resolution, and under the same grid of assumed display parameters. Our fitted metric gets no extra information beyond its fitted weights. No reported result was found on Lacuna for any of these metrics predicting the output drift of a retinal model, so the comparison has to be made here. Beating them means a higher rank correlation between metric score and $|\Delta y|$ on held-out perturbation families, with bootstrap intervals that exclude the best human-referred metric.

For the no-reference setting, the strongest existing method is MCF-Net. Evaluation of Retinal Image Quality Assessment Networks in Different Color-Spaces reports accuracy 91.75% and F-measure 0.8551 on the EyeQ test set with a DenseNet121 backbone. We retrain it on the same EyeQ split with the same backbone and tuning budget as our distilled regressor. We then use its class probabilities as a continuous quality score. Test-time augmentation variance, meaning the spread of the model's predictions over random color and brightness jitter of one image, is a further direct competitor and is treated as such.

Two ablations isolate the key ingredient. One fits $D_m$ on luminance bands only, which removes chrominance. The other keeps all channels but fixes the weights to human castleCSF values, which removes model referral. Beating MCF-Net means that, at matched rejection rates on EyeQ, images retained under our score show smaller downstream error.

Why not just…?

Why not just gate with MCF-Net or another human-graded quality classifier? MCF-Net is trained to reproduce what graders consider gradable, and its own result shows that the "Usable" grade still hides measurable diagnostic loss. A label scheme designed for retinopathy grading has no reason to rank images by how much they disturb a parenchymal age or stroke signal. This is an empirical question. The matched-rejection-rate experiment below settles it. If MCF-Net probabilities order images by downstream error as well as our score does, the gating claim fails and we will report that.

Why not just estimate per-image instability with test-time color jitter? This is the most serious threat to the gating use case, and it may well match a distilled regressor at flagging unstable images. Jitter variance has two weaknesses. It requires choosing a jitter distribution, and it cannot tell nuisance from signal. If a model's output moves under a blue–yellow shift, jitter only reports that it moved, not whether that sensitivity reflects acquisition artifact or real parenchymal pallor. The audit's synthetic ground truth is what separates the two, so we position gating as a secondary payoff. The primary contribution is the sensitivity map and the human-versus-model comparison. We will also report a version of jitter variance whose jitter is restricted to the axes the audit identifies as nuisance, to see whether the map improves it.

Why not just let foundation-model scale and color augmentation handle invariance? RETFound and DeepRETStroke (A deep learning system for detecting silent brain infarction and predicting stroke risk) were pretrained on large but demographically concentrated cohorts. Masked-autoencoder pretraining reconstructs raw pixels, so it has no built-in reason to discard acquisition color. Generic color jitter is also risky, because it may erase the chromatic parenchymal signal that the inpainting result suggests matters. The augmentation experiment tests this directly: generic jitter versus nuisance-only augmentation, evaluated across quality strata.

Why not just enhance images first? Enhancement networks such as cofe-Net normalize illumination using vessel and disc priors. Its reported gains are segmentation gains, with DRIVE vessel AUC rising from 0.924 to 0.957. It does not report whether systemic predictions or vessel widths are preserved. We include "enhance, then predict" as an arm of the audit. If enhancement itself induces drift on clean images, that is a finding.

Proposed Experiments

The first experiment is the audit and sensitivity comparison. It runs on EyeQ "Good" images, with channel–band threshold maps $T_m$ for the three readouts and a test of whether $T_m/T_h$ departs from a constant. The prediction is that low-frequency chromatic bands depart most. A flat ratio within bootstrap uncertainty would falsify the core claim.

The second experiment is the drift-prediction benchmark. All metrics are compared by rank correlation with $|\Delta y|$, using leave-one-perturbation-family-out evaluation so that the fitted metric cannot memorize a single distortion type. The decisive outcome is whether castleCSF-weighted ColorVideoVDP comes within confidence intervals of the fitted metric. If it does, human vision is an adequate proxy and the project reduces to a negative result worth publishing.

The third experiment is real-image validation on EyeQ. We stratify test images by the distilled score, by MCF-Net probability and by jitter variance, then plot downstream error against retention rate for each. The decisive check is the area between these curves. A secondary check, where a cohort with per-eye age or sex labels is accessible, is fellow-eye disagreement on targets that are identical across the two eyes. Such disagreement is acquisition error by definition, so it gives a natural experiment for which score explains it.

The fourth experiment quantifies caliber bias. We express vessel-width drift under realistic perturbation magnitudes in micrometers, using optic-disc diameter as an approximate scale. We then compare that drift with the 0.8 µm CRAE difference reported in Influence of secondhand smoke exposure on the retinal vasculature of children in Hong Kong. Random drift averages out in group comparisons. Only systematic, direction-consistent drift could confound such a study, so we report the signed mean shift separately from its spread. The conversion from pixels to micrometers is approximate and will be stated as such.