Photographs of the back of the eye can predict stroke. The retina develops from the same embryonic tissue as the brain, and its small blood vessels share much of the brain's physiology. Large retinal models trained on these photographs already beat standard clinical risk factors at finding silent brain infarctions, which are small strokes that cause no symptoms and are usually found only on MRI. What no one knows is which part of the photograph carries that signal. It could be the vessels, whose narrowing and widening track hypertension and endothelial damage. It could be the surrounding retinal tissue, called the parenchyma. Or it could be camera and clinic fingerprints that happen to correlate with who has strokes. The evidence points in several directions. A vessel-graph model that throws away every non-vessel pixel predicts stroke about as well as a full-image network. A separate study found that age and sex remain highly predictable after the vessels are digitally erased. And in a UK Biobank model, performance differed by up to 10 AUC points between assessment centres that used identical hardware.
We propose to answer the question directly. We will build a stroke predictor whose output is, by construction, a sum of a vessel term, a parenchyma term, and a clinical-traits term. We will then test each term with controlled image edits, which the model must respond to in physiologically sensible ways. The first edit erases vessels. The second transplants parenchyma from a different clinic. The third changes arteriolar caliber by a set amount, where caliber means the measured width of the small arteries. The result is a quantitative account of where retinal stroke risk lives and how much of it survives removal of site shortcuts. It also answers a practical question: does a deep model add anything beyond classical vessel measurements plus clinical traits?
Core Claim
The output of retinal stroke models can be split into vascular and parenchymal contributions that are both read off the model and verified by intervention. This split exposes whether parenchymal signal is real biology or acquisition-site leakage, and plain accuracy metrics cannot answer that question. We expect three findings. First, the vascular term transfers across assessment centres. Second, part of the parenchymal term is site-specific. Third, a model restricted to vascular plus site-robust parenchymal signal keeps almost all of the full-image model's discrimination. If the third expectation fails, and a simple model on measured vessel features plus clinical traits matches the deep model, that is a publishable negative result and we will report it as such.
Relation to Prior Work
Several papers motivate the gap. DeepRETStroke pretrains a large masked-autoencoder vision transformer on raw fundus photographs. It reports an AUC of 0.797 for silent-infarct detection against 0.633 for clinical traits, and 0.901 for incident stroke. It never shows what image content drives those predictions. RETFound is the general-purpose retinal foundation model, and it is likewise trained and evaluated on whole images. Alloula et al. found age and centre gaps in a UK Biobank hypertension classifier. They also showed that group-robust training narrows those gaps only by "levelling down", meaning it lowers performance for everyone. This is why we audit where signal comes from instead of reweighting groups.
Two papers pull in opposite directions, and our design is built around that tension. Huang et al. turn segmented vessels into graphs and reach a stroke AUROC of 71.92% on UK Biobank, against 71.54% for a raw-image ResNet-50. That near-tie suggests the vessels carry most of the signal. Kim et al. inpainted vessels away and found that parenchymal texture still predicts age and sex. That suggests a whole-retina imprint. Neither study looked at stroke with both regions measured in one model, and neither separated parenchymal biology from site artefact.
Other papers supply methods. AutoMorph provides quality grading, vessel, artery/vein and optic-disc segmentation, and standard caliber measures. These include CRAE and CRVE, which are the central retinal artery and vein equivalents, and AVR, the artery-to-vein ratio. Its artery/vein F1 is only 0.66, so we treat its masks as soft and noisy. Mask-weighted pooling from Bai gives us a way to force a feature to come only from a region. Gare et al. show in lung ultrasound that masking out a shortcut tissue raised AUC from 0.655 to 0.774, which is precedent for region control exposing shortcuts.
Other papers constrain the evaluation. Sadafi et al. showed that saliency maps that look plausible need not reflect what the model actually depends on, and they used remove-and-retrain tests to check. We therefore refuse to rely on heatmaps. The retinal diffusion counterfactual work shows that realistic fundus edits are feasible, but its edits follow classifier gradients and spread diffusely across the image. GCDM shows generation can be steered by measured clinical features. We combine the two, and the edits we make are defined in AutoMorph's measurement units rather than classifier gradients.
The closest single paper is PRETI. It restricts masking to the circular fundus and inspects attention maps. It treats the whole disc as one region, however, and it does not study stroke.
Research Program
The predictor shares one encoder, RETFound's vision transformer initialised from public weights. Its outputs are pooled separately under two soft masks. The vessel mask is AutoMorph's vessel probability map, dilated by a few pixels so that vessel walls and the perivascular region are kept. The parenchyma mask is its complement inside the fundus, with the optic disc treated as a third small region. The stroke logit is:
$$ \ell(x, c) = f_v\big(\mathrm{pool}_{m_v}(E(x))\big) + f_p\big(\mathrm{pool}_{m_p}(E(x))\big) + g(c), $$
In this equation, $E(x)$ is the encoder's patch tokens for image $x$, and $m_v$ and $m_p$ are the vessel and parenchyma masks. $\mathrm{pool}_m$ is mask-weighted average pooling, and $f_v$ and $f_p$ are small heads. $g(c)$ is a linear term on clinical traits $c$: age, sex, blood pressure, smoking, diabetes and medications.
Self-attention in the encoder lets information leak across regions, so the pooled vessel feature could still "see" parenchyma. We control for this with a region-isolated variant in which attention is masked so that vessel tokens attend only to vessel tokens. We report both variants, because the gap between them measures cross-region leakage.
The audit runs three families of edits on held-out images. Vessel erasure inpaints the vessel mask, as Kim et al. did. It is paired with an area-matched control that erases random non-vessel regions of the same area, so that simply inpainting pixels is not mistaken for losing vessel signal. Parenchyma transplantation keeps an image's vessels and optic disc and replaces its background with parenchyma from a donor matched on age and sex. The donor either comes from a different assessment centre with the same outcome, or from the same centre with the opposite outcome. That contrast separates site signal from disease signal. Caliber edits use a mask-conditioned diffusion inpainter, conditioned the way GCDM conditions on radiomic features, to narrow arterioles by set CRAE decrements. Each edit is accepted only if AutoMorph re-measures the intended change and finds CRVE unchanged within tolerance.
Faithfulness is then scored in three ways. Vessel erasure should mainly move $f_v$. Cross-centre transplantation with the outcome held fixed should not move $f_p$ much. Arteriolar narrowing should raise predicted risk, in line with the caliber literature summarised by Cheung et al..
Baselines
The naive reference is a logistic model on clinical traits alone, fit on the same UK Biobank training split. On silent infarcts in the DeepRETStroke cohorts it reached AUC 0.633. We will refit it on UK Biobank incident stroke and treat it as the floor.
The strongest interpretable competitor is AutoMorph's vessel measurements plus clinical traits, fit with regularised logistic regression and gradient-boosted trees. These measurements are tortuosity, fractal dimension, density, CRAE, CRVE and AVR by zone. This baseline gets the same folds and a hyperparameter search matched in trial count to the deep models. It is the comparison that decides whether deep features add anything at all.
The strongest deep competitor on this data is the vessel-graph contrastive model of Huang et al., which reports a UK Biobank incident-stroke AUROC of 71.92%. Its raw-image ResNet-50 counterpart reports 71.54%, and a supervised multimodal baseline 69.30%. We will reimplement it on our splits instead of borrowing its numbers.
Whole-image RETFound fine-tuned with clinical traits is the natural foundation-model baseline. I found no reported result on Lacuna for RETFound on UK Biobank incident stroke, so we will establish it ourselves with the same epochs and tuning budget as our model. A PRETI-style variant that masks only to the fundus disc tests whether coarse region control already achieves what we claim. DeepRETStroke is the highest reported number, with an incident-stroke AUC of 0.901. That figure is internal validation on different cohorts, so it is not directly comparable. We will evaluate its weights on our splits only if they are obtainable.
Two ablations isolate the key ingredients. The first is our model with a single pooled head, which removes the additive split. The second replaces soft dilated masks with hard binary masks, which tests brittleness to segmentation errors.
"Beating" the baselines is defined deliberately. Our model must match whole-image RETFound's AUROC within its bootstrap 95% interval. It must show a smaller leave-one-centre-out AUROC drop. And it must pass the intervention tests that the baselines either cannot be run on or fail.
Why not just…?
Why not just fit classical vessel measurements plus clinical traits? This is the real threat. Huang et al.'s near-tie between vessel graphs and raw images hints that little beyond vessel geometry matters. Our experiment settles it directly: if the AutoMorph-plus-traits model comes within the deep model's confidence interval under leave-one-centre-out evaluation, we will conclude that the deep model adds nothing robust. We will then report the parenchyma findings as evidence about shortcuts rather than new signal. What we expect, but do not assume, is that the deep vascular term beats the hand-measured features. The reason is that AutoMorph's artery/vein separation is weak (F1 0.66), and its summary statistics discard local detail.
Why not just fine-tune RETFound on whole images and inspect Grad-CAM? Sadafi et al. showed that attributions that look convincing can diverge from what the model actually uses. Class activation maps are also too coarse to separate a vessel from the tissue beside it. A heatmap cannot tell us whether parenchymal activation reflects biology or centre fingerprints. The transplantation test can, because it holds the vessels fixed and varies only the source of the parenchyma.
Why not just use vessel graphs and discard parenchyma? Kim et al. show that parenchyma carries systemic signal, so discarding it risks losing real information. Our parenchyma term, measured under cross-centre transplantation, will show how much of that information is disease signal and how much is site signal. If the site-robust part of the parenchyma term is near zero, the graph approach is vindicated. If not, graphs leave accuracy on the table.
Why not just repeat Kim-style vessel inpainting on a stroke model? Inpainting alone confounds removing signal with making the image unusual. A drop in performance could mean the vessels mattered or merely that the edited image is out of distribution. The area-matched random-erasure control, and the requirement that AutoMorph confirm each edit, are what separate those two readings.
Proposed Experiments
All training uses UK Biobank fundus photographs with incident stroke from hospital records, the cohort size Huang et al. used (82,569 individuals). Folds are split by participant. Images that fail AutoMorph quality grading are excluded, and we will report how many. Hypertension is a secondary endpoint, which lets us compare with Alloula et al.'s 71% AUC and their centre gaps. External checking uses Moorfields AlzEye linked outcomes if access is granted. Otherwise we will state that limitation explicitly.
The primary metrics are AUROC with bootstrap intervals, the worst-centre AUROC under leave-one-centre-out, and calibration. For the audit we measure three things. The first is the shift in each term's contribution, $f_v$ and $f_p$, per edit, relative to area-matched controls. The second is the proportion of cross-centre transplants that change predicted risk by more than a clinically meaningful threshold. The third is the dose-response slope of predicted risk against edited CRAE.
A blinded reader study will check that edits are realistic, using the odd-one-out design of the retinal diffusion counterfactual work. It is sized to what two ophthalmologists can grade, and it guards against the generator and the classifier sharing blind spots.
The decisive checks are four. Does the site-robust model retain whole-image discrimination? Does the parenchyma term survive centre transplantation? Does arteriolar narrowing raise risk in the expected direction? Does any deep model beat measured vessel features plus clinical traits across centres? Each outcome, positive or negative, changes how retinal stroke screening should be built and trusted.
References
- A deep learning system for detecting silent brain infarction and predicting stroke risk
- A foundation model for generalizable disease detection from retinal images
- On Biases in a UK Biobank-Based Retinal Image Classification Model
- Predicting Stroke Through Retinal Graphs and Multimodal Self-supervised Learning
- Effects of Hypertension, Diabetes, and Smoking on Age and Sex Prediction from Retinal Fundus Images
- AutoMorph: Automated Retinal Vascular Morphology Quantification Via a Deep Learning Pipeline
- Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling
- The Role of Pleura and Adipose in Lung Ultrasound AI
- Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Images
- Development and validation of an AI algorithm to generate realistic and meaningful counterfactuals for retinal imaging based on diffusion models
- Joint Holistic and Lesion Controllable Mammogram Synthesis via Gated Conditional Diffusion Model
- PRETI: Patient-Aware Retinal Foundation Model via Metadata-Guided Representation Learning
- Influence of secondhand smoke exposure on the retinal vasculature of children in Hong Kong