Teaching Retinal Stroke Screeners Across Clinics That Never Scanned a Brain

A photograph of the back of the eye shows small blood vessels that develop alongside, and behave much like, the vessels of the brain. Deep networks trained on these photographs can now flag silent brain infarctions, which are small strokes visible only on brain MRI, and can predict future stroke better than blood pressure, age and smoking combined. Training such a screener, however, currently requires one institution to hold hundreds of thousands of retinal photographs linked to brain scans. The clinics where retinal screening would matter most are community eye clinics and primary-care sites. They have fundus cameras and stroke records but no MRI, and privacy rules prevent them from sending their images anywhere.

We propose asymmetric eye-to-brain federated transfer. In federated learning, sites train a shared model by exchanging parameter updates instead of raw images. Here, one MRI-equipped hub holds retina–brain paired labels, and many camera-only satellite clinics hold unlabeled photographs plus routinely recorded stroke outcomes. The hub's infarct detector labels satellite photographs locally. Each satellite first absorbs its own camera's quirks in a small private adapter, so that the hub's labels stay meaningful on its images. Local stroke outcomes act as a check on those labels. Only small shared adapter weights, protected with differential privacy, ever leave a site. The central claim is testable: under realistic camera and population shift, this beats both shipping the hub's model unchanged and plain federated averaging on silent-infarct detection and stroke discrimination at satellite sites. It does so without degrading the fine vessel geometry the task depends on, which image-anonymization approaches do degrade.

What exists, and what is missing

Two recent systems establish that the retinal signal is real. RETFound is a vision transformer pretrained by masked autoencoding, meaning it learns by reconstructing hidden patches of 1.6 million retinal images. It predicts 3-year heart failure with an area under the ROC curve (AUROC) of 0.794 from colour fundus photographs. DeepRETStroke supplies the mechanism we reuse. After retinal pretraining, it uses an MRI-confirmed infarct detector to produce probabilistic "soft labels" on retinal images that have no MRI, and trains jointly on infarct and incident-stroke targets. It reports an AUC of 0.797 for silent infarcts against 0.633 for a clinical-trait model, and 0.901 for incident stroke. Both are centralized, and both were built on demographically narrow cohorts. The phenome-wide retinal atlas adds a constraint: retinal images did not improve gross cardiovascular prediction over clinical scores such as QRISK3. We therefore target cerebral small-vessel endpoints, where the retinal signal is not redundant, and we always report gains over clinical traits.

Federated retinal work so far is confined to eye disease. FedUAA weights client updates by evidential uncertainty for diabetic retinopathy staging. Fair-FedMOE routes samples to demographic experts in a single communication round for glaucoma and retinopathy. FedIAT personalizes the input side of low-rank adapters to absorb scanner shift while sharing the output side, for optic disc segmentation. All three assume every client holds labels for the same task. None addresses systemic or cerebral targets, and a review of federated ocular imaging explicitly lists systemic oculomics as open.

Semi-supervised federated methods handle unlabeled clients: FAT alternates teacher pseudo-labeling on unannotated silos, and FLowDUP personalizes unlabeled clients in a low-dimensional subspace. In both, labels and pseudo-labels come from the same modality and task. They do not cover our case, where the gold label comes from a different organ's imaging available at only one site, and satellites hold a different but causally related label: stroke events.

On the data-sharing side, k-SALSA shows that fundus vessel patterns identify individuals. It releases k-anonymous synthetic averages that cut membership-inference success to about 1% at k=10. It preserves lesions as local texture, though, and averaging k vessel trees plausibly erases the caliber and tortuosity signals that Cheung et al. tie to cerebrovascular risk. We treat this as a hypothesis to measure, not a given. Gim et al. argue the regulatory case that pooling retinal images may become impossible if they are classed as biometrics, which motivates designs where images never leave the site.

Core Claim

A hub-trained retinal infarct detector's soft labels degrade at satellite clinics mainly because of camera and acquisition shift, not because the eye–brain relationship differs. The claim has two parts. First, a private per-site input adapter can absorb that shift, so that the hub's labels, recomputed through the adapted encoder, become reliable again. Second, satellite stroke outcomes can then correct residual label error without anyone at the satellite ever seeing a brain scan. If the claim holds, satellites contribute real supervision to a shared cerebral screener. If it fails, federation adds nothing over shipping the hub model, and we will know that too.

Research Program

The backbone is RETFound's pretrained vision transformer, kept frozen and fitted with low-rank adapters. Following FedIAT's covariate-versus-concept split, each adapter has two factors. The factor on the input side of each block stays private to its site and models that site's camera. The factor on the output side is shared, together with a small head producing an infarct probability and a stroke hazard.

Training runs in three phases. The hub first trains the shared factors and head on its MRI-paired data, using DeepRETStroke's recipe. Each satellite then fits only its private input factors, using a masked-reconstruction loss on its own unlabeled photographs, so the encoder learns the local camera without any label. Finally, satellites compute soft infarct labels $q(x)$ through the adapted encoder and train the shared factors on

$$L_s = \mathbb{E}_{x}\big[w(x)\,\mathrm{BCE}(p_\theta(x), q(x))\big] + \lambda\, \mathrm{Cox}_s(h_\theta),$$

where $p_\theta$ and $h_\theta$ are the shared model's infarct probability and stroke hazard, $\mathrm{Cox}_s$ is the Cox partial-likelihood loss on the satellite's recorded stroke events, and $\lambda$ balances the two terms. The weight $w(x)$ down-weights soft labels whose confidence disagrees with the local stroke model, for example a confident "no infarct" in a patient who later has a stroke.

Shared-factor updates are clipped and noised for differential privacy and aggregated by weighted averaging. We will run in both multi-round and few-round regimes, because clinics have limited bandwidth. The analysis side asks where the transferable signal lives. We measure how much hub-to-satellite AUC drop is recovered by the private factors alone. We also use vessel segmentation tooling (SGL vessel segmentation) to test whether errors concentrate in images where caliber estimates are unstable.

Baselines

The naive reference point is the clinical-trait model: age, sex, blood pressure, smoking, diabetes and lipids, fitted by federated logistic or Cox regression. Tabular federation is easy. On silent infarcts, DeepRETStroke reports this class of model at AUC 0.633 against 0.797 for retinal deep learning, in a centralized setting.

The strongest practical competitor is hub-only transfer. DeepRETStroke is trained at the hub and shipped to satellites unchanged. A second version lets each satellite fine-tune it on its own stroke outcomes with no parameter sharing. This is what a hospital would actually deploy. Every other retinal baseline is also given this option of fine-tuning on satellite stroke labels, so that the comparison isolates the federated soft-label mechanism rather than access to satellite labels.

The standard federated competitor is plain federated averaging of full low-rank adapters. Hub and satellites participate on equal footing, satellites train on stroke labels only, and nothing is personalized. We also include FAT-style alternating mean-teacher pseudo-labeling, and FedUAA's uncertainty-weighted aggregation adapted to a binary infarct head.

The upper reference is centralized pooling of all images with DeepRETStroke's recipe, the configuration that privacy rules forbid. No reported result was found on Lacuna for any federated method on silent infarct or stroke prediction. DeepRETStroke's centralized incident-stroke AUC of 0.901 is on its own cohort and is not directly comparable to ours. It frames the ceiling.

All methods share the same RETFound initialization, the same adapter rank budget, the same number of client-side gradient steps, and an equal hyperparameter search budget per method. Search is done on hub validation data only, so that no method tunes on satellite test labels.

Two ablations isolate the key ingredients. The first shares the input factors instead of keeping them private. The second removes the stroke-outcome weighting, setting $w \equiv 1$. Beating the competitors means a statistically significant gain in satellite-site silent-infarct AUC over hub-only transfer with local fine-tuning, confirmed by bootstrap confidence intervals over patients. It also means recovering at least half the gap between hub-only transfer and centralized pooling, without losing stroke C-index relative to any baseline.

Why not just…?

Why not just train at the hub and ship the model? This is the real threat. If foundation pretraining already makes the representation camera-invariant, shipping will match federation. We do not argue this away; the hub-only arm is the main comparison. The plan also includes a direct diagnostic: the drop in infarct AUC from hub to satellites under shipping. If that drop is negligible, we will report that federation is unnecessary for this task. That would itself be useful, because it would mean privacy-preserving deployment is simple.

Why not just pool k-anonymized synthetic images? k-SALSA solves identifiability but builds each synthetic image from k patients' vessel trees. We test this directly with three measurements on real versus k-SALSA images: vessel caliber and tortuosity fidelity, infarct AUC of a model trained on each image set, and membership-inference success for both k-SALSA and our differentially private adapters. If synthetic pooling preserves infarct AUC at comparable privacy, it is a simpler route, and we will say so. Its reported strengths are fidelity by FID and robustness to membership inference on retinopathy grading, which does not show that it preserves vascular geometry.

Why not just share vessel measurements as tabular data? Summary caliber and tortuosity statistics are low-dimensional, carry little identifying information, and are trivial to federate. This baseline, run on top of clinical traits, will be included. Kim et al. found that retinal tissue retains systemic signal after vessels are removed. DeepRETStroke's deep model also beats trait models. Together these suggest hand-crafted vessel metrics leave signal unused, but how much is an empirical question, and we will measure the gap.

Why not just use plain federated averaging? On retinopathy, FedUAA reports AUC 0.8636 against 0.8484 for FedProx and 0.8519 for FedBN, and FedIAT reports a 27% gain over uniform federated low-rank tuning on an out-of-distribution fundus client. Both show that uniform averaging suffers under camera shift. Plain averaging also gives satellites no channel for the hub's infarct supervision. It is included as a baseline.

Proposed Experiments

The primary testbed is a simulated federation built from UK Biobank fundus photographs linked to incident stroke records over the atlas study's median 11.4-year follow-up. Assessment-centre partitions serve as satellites. A hub partition holds brain-imaging-derived infarct labels, drawn from UK Biobank participants with brain MRI or, failing that, from a collaborating DeepRETStroke site. Deep-learning methods will also be evaluated against the clinical-trait model built from the same cohort's recorded risk factors.

Real camera shift will be introduced by adding satellites drawn from public fundus datasets with different devices and populations: APTOS 2019, EyePACS, Messidor, IDRiD, DDR and BRSET. These carry no stroke labels, so they are used only for the shift diagnostics and the reconstruction-based adapter phase. External validation of stroke discrimination uses EPIC-Norfolk and, if access permits, Moorfields AlzEye.

Metrics are silent-infarct AUC at the hub's held-out satellite-like data, incident-stroke C-index at each satellite, and worst-site performance. Fair-FedMOE's worst-subgroup AUC is used where demographics are recorded.

The decisive checks are four. The first is whether the private-factor ablation reproduces most of the gain: if not, the camera-shift premise is wrong. The second is whether gains grow with measured camera shift across satellites. The third is whether the stroke-outcome weighting improves infarct AUC, rather than merely stroke C-index. The fourth is whether, at matched membership-inference success, our shared adapters preserve downstream infarct AUC better than k-SALSA-pooled images.