Retinal photographs are cheap, and models trained on them can flag people at risk of stroke and dementia. Most of these models learn from hospital diagnosis codes linked to eye-clinic images. The codes are treated as truth, but for cerebrovascular disease they are wrong in a particular, one-sided way. Silent brain infarcts and small vessel disease are, by definition, rarely diagnosed. Coding also differs between hospitals, between age groups, and with how long a patient has been followed. As a result, the "no stroke" class is filled with people who have real cerebral vessel damage. The retina may be showing exactly the signal the model is being taught to ignore.
This note proposes to treat each hospital's coding practice as one "observer" with its own sensitivity, borrowing the inter-observer modeling idea from pathology. The retinal model is then trained against a learned account of who gets coded, rather than against the codes themselves. The bet is narrow and testable. If coding sensitivity varies with patient covariates (hospital trust, age, deprivation, follow-up length), then training on raw codes distorts the risk ranking across subgroups, not just the calibration. Correcting for that variation should make retinal scores track brain-imaging measures of vessel disease more closely. If sensitivity turns out to be uniform, the correction collapses to simple recalibration, and the experiments are designed to reveal that.
Core Claim
Two facts in the literature point to the same problem. First, the AlzEye linkage of 6.26 million images from 353,157 Moorfields patients to Hospital Episode Statistics (AlzEye) supplies outcome labels for 11,735 stroke and 13,363 dementia patients. Its authors say these administrative codes suffer from miscoding, omission, and variation across hospital trusts. Second, DeepRETStroke (A deep learning system for detecting silent brain infarction and predicting stroke risk) shows that retinal photographs detect MRI-confirmed silent brain infarction with AUC 0.797, against 0.633 for clinical-trait models. Put together, a sizeable share of code-negative patients plausibly carry retinally visible cerebrovascular disease that no code will ever record.
Generic noisy-label methods are poorly matched to this situation. LiNC's per-sample trust (LiNC) and the Signed Entropy Integral (On Revisiting Entropy for Identifying Mislabeled Images) were validated on symmetric, synthetically injected noise with hard labels. They would "correct" a silent-disease negative into a stroke positive, which is also false. QMix (QMix) adds a further warning: in fundus images, the rule that low-loss samples are clean breaks down, because degraded images can have low loss. The subtle, diffuse signal that cerebral prediction relies on is the kind a loss-based filter might discard.
What is new here is the noise model, not the classifier. Let $y$ be latent cerebrovascular disease, $\tilde y$ the observed code, $x$ the retinal image, and $g$ the coding context (trust, age band, follow-up years). We model
$$p(\tilde y = 1 \mid x, g) = s(g)\, f_\theta(x) + \epsilon(g)\,\bigl(1 - f_\theta(x)\bigr),$$
where $f_\theta(x)$ is the retinal network's estimate of $p(y=1 \mid x)$, $s(g)$ is the coding sensitivity in context $g$, and $\epsilon(g)$ is a small false-coding rate.
A useful fact sets the stakes. If $s$ and $\epsilon$ are constant, the noisy posterior is a monotone transform of the clean one. In that case, area under the ROC curve (AUROC) trained on raw codes is already rank-optimal and only calibration suffers. The correction can only improve discrimination if $s(g)$ genuinely varies and $g$ correlates with retinal appearance. Testing that condition is the first experiment, not an assumption.
Research Program
The context-specific sensitivity $s(g)$ is not identifiable from codes alone. We pin it down with two sources of evidence the data already contain.
The first is temporal revelation. AlzEye spans 2008–2018, so a label frozen at an early cutoff can be compared with codes appearing later. Patients coded negative at the cutoff but positive within the following years are observed omission or delay errors. Their rate, broken down by context $g$, estimates part of $1 - s(g)$.
The second is cross-trust overlap. Some patients have admissions in more than one trust, which gives several institutional "observers" for one person. This is the structure the Latent Doctor Model (The Latent Doctor Model for Modeling Inter-Observer Variability) exploits for pathologists. That model encodes each rater's past annotations into a low-dimensional style vector. Here the rater is a trust and the style is its coding sensitivity.
The Latent Doctor Model learns annotator style from many raters labeling the same images. We instead learn it from sparse overlap plus time, and for an outcome no rater ever sees directly. Unlike rater-specific decoders such as OMBA-UNet (Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks), which add per-rater parameters at inference time, our observer model is used only during training. At deployment the model outputs $f_\theta(x)$ alone.
The retinal backbone is RETFound (A foundation model for generalizable disease detection from retinal images), a masked-autoencoder model pretrained on 1.6 million retinal images. RETFound is fine-tuned by maximizing the likelihood of observed codes under the equation above. A small network maps $g$ to $s(g)$ and $\epsilon(g)$, fitted jointly with $f_\theta$. Temporal-revelation and cross-trust counts enter as auxiliary likelihood terms on $s(g)$.
One caveat concerns pretraining overlap. RETFound's pretraining used MEH-MIDAS, which comes from the same hospital as AlzEye. That pretraining was self-supervised, without labels, so no outcome information leaks. Even so, every comparison uses the same backbone so that any overlap affects all arms equally.
Baselines
The naive reference is RETFound fine-tuned with cross-entropy on raw HES stroke codes, using a fixed outcome window. This is current practice, and every other arm uses the same backbone, data splits, augmentation, and hyperparameter search budget. For context, the file gives no reported result on Lacuna for RETFound stroke prediction on AlzEye. The nearest RETFound systemic figure is 3-year heart failure AUROC of 0.794 (colour fundus) and 0.799 (OCT) (Zhou et al.). The strongest reported retinal stroke figures come from DeepRETStroke: internal incident-stroke AUC 0.901 and silent-infarct AUC 0.797. That training pipeline differed (multi-national cohorts, MRI-guided transfer), so these numbers are reference points, not a matched comparison.
The strongest generic noisy-label competitors are LiNC and the Signed Entropy Integral. LiNC reports 0.9837 noise-detection AUC on OrganSMNIST at 20% synthetic noise, against 0.8652 for Area Under the Margin. The Signed Entropy Integral improves mislabel-detection F1 by more than 8% at 50% noise on ISIC 2018. Both are run on the same RETFound fine-tuning with their published schedules and an equal tuning budget.
A QMix-style two-dimensional mixture over loss and entropy is added as the quality-aware competitor. QMix reports 80.8% kappa on DRTiD. All three generic methods are scored on two things: how well they detect temporally revealed omissions, and the downstream measures below.
The key ablation is our model with $s$ and $\epsilon$ held constant across contexts. By the monotonicity argument, this arm should match the naive baseline in AUROC. Any gain of the full model over it isolates the covariate-dependent part of the noise.
Beating the baselines means four things together. First, higher AUROC against later-revealed outcomes. Second, a smaller AUROC gap between trust and age subgroups. Third, stronger partial correlation with MRI vessel-disease volumes on the external cohort described below. Fourth, confidence intervals that exclude the constant-sensitivity ablation.
Why not just…?
Why not just recalibrate the naive model afterward? If coding sensitivity is uniform, this is the right answer, and our method reduces to it. The constant-sensitivity ablation is built to show this. Recalibration rescales scores but cannot reorder patients across trusts with different coding habits. So we first estimate $s(g)$ from temporal revelation. We commit to the full method only if sensitivity varies meaningfully across contexts. If it does not, the paper's finding is that recalibration suffices, which is still useful to report.
Why not just use survival analysis with longer follow-up? Censoring-aware models handle delayed diagnosis well and are a real threat for the delay component. They cannot handle disease that is never coded. Silent infarcts that DeepRETStroke detects at 0.797 AUC never become admissions. We run a Cox-style survival head as an extra arm. We expect it to match our method on delayed-onset patients and to fall short on MRI-based measures of silent disease.
Why not just apply LiNC or the Signed Entropy Integral off the shelf? Both were validated on symmetric injected noise, and the Signed Entropy Integral assumes hard labels. A sample-level flip turns a silent-disease negative into a stroke positive, which replaces one error with another. QMix also shows that loss-based selection misfires on degraded fundus images. The temporal-revelation test measures directly whether these detectors find real omissions or mainly flag cataract-degraded images. We report detection AUC stratified by image quality.
Why not just restrict training to the trusts with the best coding? This discards data and enriches the training set for particular referral populations. That selection problem echoes the 32% image exclusion in older adults reported by Ballerini et al.. We include it as an arm, trained on the top tercile of trusts by estimated sensitivity, to measure the cost.
Proposed Experiments
The first check asks whether the noise depends on covariates. On AlzEye, we freeze labels at successive cutoffs and estimate the omission rate by trust, age, and follow-up length. We also measure code concordance among patients seen at more than one trust. If the variation is negligible, the program stops at recalibration.
The second check is real-noise detection. Using the 2008–2018 span, we hold out codes revealed after the cutoff. We then compare which code-negative patients each method identifies as likely positive, scoring with AUROC against later revelation and stratifying by image quality.
The third and decisive check is external alignment with brain imaging. We apply each frozen model to the Lothian Birth Cohort 1936. That cohort has white matter hyperintensity (WMH) volume for 478 subjects and enlarged perivascular space (PVS) volume for 381 subjects, from MRI. We compute the partial Spearman correlation of retinal scores with each volume, adjusting for age and hypertension.
Ballerini et al. found venular and calibre features selected for PVS in up to 99.6% of bootstrap trials, but retinal features selected for WMH in only about 23%. We therefore predict gains mainly on PVS. A gain only on WMH would make us suspect confounding. No deep-model correlation figure on this cohort was found on Lacuna, so every arm is computed fresh.
The fourth check is external transfer. On UK Biobank, the ICD-coded stroke model is assessed for AUROC within ethnicity and age subgroups. UK Biobank is 91.1% White European (Detection of anaemia from retinal fundus images via deeplearning), so subgroup intervals are reported with their width.
If access to DeepRETStroke's MRI-confirmed silent-infarct cohorts can be arranged, they would provide the cleanest final test. The prediction there is that the corrected model's scores separate MRI-positive from MRI-negative people among code-negatives better than the naive model's scores do.
References
- AlzEye: longitudinal record-level linkage of ophthalmic imaging and hospital admissions of 353 157 patients in London, UK
- A deep learning system for detecting silent brain infarction and predicting stroke risk
- LiNC: Lightweight Noise Correction via Adaptive Label Refinement
- On Revisiting Entropy for Identifying Mislabeled Images
- QMix: Quality-Aware Learning With Mixed Noise for Robust Retinal Disease Diagnosis
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks
- A foundation model for generalizable disease detection from retinal images
- Retinal Biomarkers Discovery for Cerebral Small Vessel Disease in an Older Population
- Detection of anaemia from retinal fundus images via deeplearning