Does the Retinal Vessel Tree Carry Information About the Brain's Structure–Function Wiring?

A photograph of the back of the eye costs a few dollars and takes seconds, while a brain scan that maps both the physical wiring (diffusion MRI) and the coordinated activity (resting-state functional MRI) costs hours of scanner time. Retinal imaging already predicts strokes and small brain lesions, so a natural question follows. Does the retina also tell us something about a specific person's brain wiring, beyond the population average and beyond what age, sex and blood pressure already explain? We propose to test this directly. We represent each retina as a tree of blood vessels at several scales and ask whether that tree predicts a quantity no one has targeted from the eye: how tightly each brain region's activity follows its anatomical connections, called structure–function coupling. A model that only sees the average brain, or only sees a person's age, must not be able to make the same prediction.

The work matters for a practical reason and for a scientific one. Paired eye-and-brain data are scarce. UK Biobank has about 6,000 people with both, and such records cannot be pooled freely across institutions. The answer also decides whether "oculomics", meaning reading systemic disease from eye images, can say anything about brain function or only about coarse lesions. We treat the hierarchical vessel representation as the mechanism under test, brain connectivity as the target, and data scarcity and privacy as constraints that shape the design. Privacy also gives a secondary aim with an explicit stopping rule.

Core Claim

Our claim is that a retinal representation which keeps the nesting of vessels (large arcades, then branch segments, then local capillary-scale patches) predicts individual, covariate-adjusted deviations in regional structure–function coupling. It should do so better than the same image features pooled without that nesting. The signal should be anatomically specific: strongest along the visual pathway and in regions prone to small-vessel disease, weak elsewhere.

Current work leaves this open. Eye-brain connections revealed by multimodal retinal and brain imaging genetics in the UK Biobank is the closest study. It correlated 156 global retinal traits with 458 brain MRI traits, found 625 associations, including ganglion-cell layer thickness with posterior thalamic radiation integrity, and showed that adding retinal traits raised fluid-intelligence prediction from r = 0.314 to r = 0.344. However, it treats the retina as flat summary vectors and the brain as separate univariate traits. It never asks about the relationship between a region's wiring and its activity.

Other studies target static lesions. Retinal Biomarkers Discovery for Cerebral Small Vessel Disease in an Older Population linked handcrafted vessel measurements to white-matter lesions and perivascular spaces. A deep learning system for detecting silent brain infarction and predicting stroke risk mapped whole photographs to infarct labels.

Our motivation for the coupling target comes from evidence that functional disconnection can come before structural damage. Precuneus Dysfunction in Parkinson's Disease With Mild Cognitive Impairment shows this. Joint Graph Convolution for Analyzing Brain Structural and Functional Connectome also found that structure–function tethering varies by region. A vascular supply problem visible in the eye is a plausible cause of such decoupling. That makes coupling, rather than lesion volume, the target where a vascular proxy should add the most.

Research Program

The retinal representation. We segment the vessels in each fundus photograph and build a three-level graph.

  • Top level: nodes for the major arteriolar and venular arcades.
  • Middle level: nodes for individual branch segments between bifurcations. Each carries geometry: calibre, tortuosity, branching angle and local fractal dimension.
  • Bottom level: image patches centred on each segment. Each carries features from the frozen RETFound encoder (A foundation model for generalizable disease detection from retinal images).

Edges come in two kinds. Edges within a level connect vessels that are physically adjacent. Edges across levels connect each patch to the segment that contains it, and each segment to its arcade.

The method we reuse. The two-tier message passing (within a scale, then across nested scales) comes from A Graph-Based Multi-Scale Approach With Knowledge Distillation for WSI Classification and Diagnose Like A Pathologist. Both showed that explicit containment beats flat multi-resolution pooling in histology slides. We transfer this to vessel trees, where nesting is a physical fact rather than a magnification convention. Because the patch features are RETFound's, any gain over a flat RETFound baseline can be attributed to the tree structure and not to a better encoder.

A parallel texture stream. We add a stream that sees only the image with vessels removed. Kim et al. showed that this non-vessel tissue still encodes age and sex. Without this control, a "vessel" result could be texture in disguise.

The brain target. In UK Biobank participants with fundus photographs, diffusion MRI and resting-state fMRI, we parcellate the brain into regions. For subject $s$ and region $i$, the coupling is

$$ \kappa_{s,i} = \rho\big(\mathrm{SC}_{s,i\cdot},\ \mathrm{FC}_{s,i\cdot}\big), $$

where $\mathrm{SC}_{s,i\cdot}$ is the row of the subject's structural connectivity matrix for region $i$, $\mathrm{FC}_{s,i\cdot}$ is the matching row of functional connectivity, and $\rho$ is Spearman rank correlation. We then regress out age, sex, systolic blood pressure, BMI, diabetes, head motion and scanner site, using models fit on training folds only. The residual $\tilde\kappa_{s,i}$ is the prediction target.

Residualizing is the central design choice. It stops the model from scoring well by reproducing the average brain or by inferring age from the eye.

Secondary aim: a shareable synthetic resource. Paired eye–brain records are scarce and access-restricted. So we will train a conditional diffusion model, a generator that learns to turn random noise into samples by gradual denoising. It produces residual coupling vectors given a retinal graph embedding, inside the approved data environment. Only synthetic pairs would leave that environment.

We evaluate the synthetic pairs on the four axes of You Don't Have to Be Perfect to Be Amazing: fidelity, variety, privacy against membership inference, and utility. The diffusion choice follows A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks. There, classifiers trained on synthetic fundus images reached 0.88 AUROC versus 0.89 for real data, while GANs were less diverse.

This aim is conditional. It runs only if the first aim finds signal, because a generator conditioned on an uninformative retina just produces noise around zero.

Baselines

Naive reference. We predict zero residual coupling, which means "this person's coupling is exactly what age, sex, vascular risk factors and site imply." This is the population-average brain after covariates. Any retinal model must beat it, and beating it is the whole scientific claim. No reported result was found on Lacuna for retina-to-coupling prediction against such a reference. We also report raw, unresidualized coupling prediction against a population-mean predictor. This shows how much apparent accuracy in cross-modal synthesis is the shared group pattern. The reported 0.731 Pearson correlation for structural-MRI-to-functional-network synthesis in Cross-Modal Synthesis of Structural MRI and Functional Connectivity Networks via Conditional ViT-GANs is not reported against a mean-matrix reference in the file, which is why we refuse to use raw correlation as the headline metric.

Covariates only. A gradient-boosted model on the clinical covariates plus additional risk markers (cholesterol, HbA1c, smoking) predicts the residual. This checks that residualization truly removed covariate signal. A deep learning system for detecting silent brain infarction and predicting stroke risk reports that clinical traits alone reached AUC 0.633 for silent infarcts against 0.797 from retinal images. That is the closest evidence that eye images add beyond risk factors, though for a different target.

Strongest existing method. RETFound (A foundation model for generalizable disease detection from retinal images) is fine-tuned end to end with a ridge or attention-pooling head to regress the residual coupling vector. It is the strongest competitor because it already beats task-specific models on systemic prediction, for example heart-failure AUROC of 0.794 from fundus photographs, and it sees every pixel our model sees. It gets the same folds and the same hyperparameter search budget, measured in GPU-hours, as our model. Beating it means a statistically reliable gain in held-out explained variance of $\tilde\kappa$, by paired bootstrap over subjects, and not just matching it.

Handcrafted geometry. Elastic net on VAMPIRE-style vessel measurements, following Retinal Biomarkers Discovery for Cerebral Small Vessel Disease in an Older Population, with the same 500-resample bootstrap for feature stability.

Key ablations. The first uses identical RETFound patch features with all cross-level edges removed, which is flat pooling. The second keeps the graph but randomly reassigns patches to segments, which preserves node count and degree but destroys physical nesting. If the true tree does not beat the shuffled tree, the hierarchy claim fails. We also run vessel-only and texture-only streams separately.

Secondary aim baselines. The diffusion generator is compared with a Gaussian copula fit to the same paired vectors and with a heteroscedastic regression, meaning a predictor of both mean and per-person variance. The comparison covers distributional calibration (continuous ranked probability score and interval coverage), utility when a downstream regressor is trained on synthetic pairs and tested on real ones, and membership-inference AUC.

Why not just…?

Why not just fine-tune RETFound and skip the vessel graph? This is the most serious threat. It may well match our model, because masked-autoencoder features can implicitly encode branching. The shuffled-tree and flat-pooling ablations settle it. All three variants share RETFound features, so if the true tree gives no gain over flat pooling, we will report that hierarchy adds nothing for this target. In that case the proposal reduces to its target-definition contribution, which still stands. The histology evidence that containment helps comes from gigapixel slides with thousands of patches, a regime quite unlike a single fundus photograph, so we treat that evidence as motivation, not assurance. The outcome where hierarchy fails is plausible.

Why not just use handcrafted vessel measurements with a linear model? In older adults, Retinal Biomarkers Discovery for Cerebral Small Vessel Disease in an Older Population found retinal measurements selected in only about 23% of bootstrap trials for white-matter lesions, and usable measurements in only 68% of participants. Both point to fragile signal and heavy data loss. Our graph is built from a learned segmentation and degrades gracefully on poor images. We will report performance on exactly the subset where handcrafted measurement fails. If the elastic net matches us on the subset where it works, the simpler tool wins there and we will say so.

Why not just synthesize the full functional connectivity matrix from the retina, as was done from structural MRI? A generator trained on matrices that share a strong group pattern can score high correlation by reproducing that pattern. A retina is a far weaker conditioning signal than a brain scan. We therefore target covariate-residualized coupling and report the population-mean reference beside any raw-matrix number. Full-matrix synthesis is a later step, justified only if the residual signal exists.

Why not just share model weights or train across sites instead of releasing synthetic data? For a single prediction task, sharing a model is simpler and we would recommend it. Synthetic pairs are useful only for questions not fixed in advance. The secondary aim is therefore judged on cross-task utility and privacy against the copula and regression baselines, and it is dropped if neither condition below is met.

Decisive Checks

Primary data. We use the roughly 6,000 UK Biobank participants with concurrent eye and brain imaging. Splits are made by participant with five-fold cross-validation, and the final held-out fold is untouched until the end.

Primary metric. The headline number is the cross-validated $R^2$ of residual coupling, averaged over regions, with paired bootstrap confidence intervals against each baseline. Region-wise results are corrected for multiple comparisons.

Specificity test. This is preregistered. Retinal predictability should be highest for visual-pathway regions (lateral geniculate and thalamus, occipital cortex), consistent with the ganglion-cell–thalamic radiation link in Eye-brain connections revealed by multimodal retinal and brain imaging genetics in the UK Biobank, and for periventricular white matter. It should be near zero in regions without such a link. A uniform signal across all regions would suggest a global confound such as image quality or residual age. We will additionally probe for that by predicting image-quality scores from the residuals.

Positive control. The same pipeline must recover white-matter lesion volume and perivascular-space burden as targets. If it cannot recover the known lesion relationships reported in the cited literature, failure on coupling is uninterpretable.

External check. We use the Lothian Birth Cohort 1936 retina–MRI pairs only for this lesion positive control. No external cohort with retina, diffusion MRI and fMRI is available in the file, so the coupling result is a single-cohort finding with a European-ancestry bias. We state that limitation plainly.

Secondary aim pass condition. This aim proceeds only if the primary residual signal is significant. It then passes only if two conditions hold. First, a regressor trained on synthetic pairs retains at least most of the real-data explained variance on the real held-out fold. Second, membership-inference AUC stays near chance, compared with the copula and heteroscedastic-regression alternatives.