Some people carry heavy brain damage and still think clearly. Researchers call this cognitive reserve. Measuring it is hard because it is defined by a mismatch, and most studies stand in for it with years of schooling. We propose a cleaner measurement in two steps. First, use a brain age model to say how damaged a brain looks. This is a regression trained only on healthy people to predict age from MRI, so its error on a new person reads as "this brain looks older or younger than it should." The model never sees cognitive scores. Second, treat reserve as whatever is left over: how much better or worse a person performs than their brain's apparent age predicts. We then ask whether a language model reading ordinary clinical notes can recover the life history that explains this leftover. That history includes occupation, literacy, bilingualism, and social engagement, which structured fields rarely capture.
Two failure modes have to be handled from the start, because either one makes the whole measurement meaningless. Brain age estimates shift with scanner and site. A site whose scanner makes brains look older will manufacture spurious "high reserve" patients. Note extraction can only find what clinicians wrote down, and wealthier, more articulate patients may simply get longer notes. So cross-dataset validation is built into the definition of the residual, not added at the end. A reserve score that does not survive moving to a new hospital, or that tracks note length, does not count as a reserve score.
Core Claim
We claim that a reserve residual defined against a normative, cognition-blind brain model and fitted with whole sites held out behaves like reserve, and that note-extracted lifelong-enrichment variables explain more of it than years of education do. "Behaves like reserve" has a specific meaning here. The residual should be stable across sites and uncorrelated with scanner identity. It should be predicted by enrichment history. It should also reproduce the signature "education paradox" reported by Vipin et al.: that study, in a Southeast Asian memory clinic, found that higher education predicts better baseline cognition but faster decline once impairment begins ($\beta = -0.13$, $p = 0.0006$ in young-onset dementia). If the residual does not show faster post-diagnosis decline among high-residual patients, it is probably measuring something other than reserve.
Formally, for person $i$ with chronological age $a_i$, cognitive score $c_i$, and a vector $r_i$ of regional brain-age residuals from the healthy-trained model, we define
$$\rho_i = c_i - \hat f_{-s(i)}(r_i, a_i),$$
where $\hat f_{-s(i)}$ is a model of cognition given brain state, fitted on every site except person $i$'s own site $s(i)$. The test is whether note-extracted reserve features $z_i$ predict $\rho_i$, and the interaction of $z_i$ with time since diagnosis predicts the cognitive slope, beyond structured education.
Relation to Prior Work
The closest work is Chen et al.. They fit a multimodal brain age gap on Cam-CAN and show that it mediates 39.6% ($p=0.017$) of the link between modifiable risk factors and cognition. That study motivates ours, but it is cross-sectional and single-cohort. Its risk factors come from questionnaires, and it treats the brain age gap as a mediator of risk, not as the denominator against which reserve is measured. Petersen et al. also motivates the design. It ran DeepBrainNet (MAE about 5.3 years) on people with and without HIV and found that reading proficiency, not years of education, related to younger-looking brains (−0.45 years per point). This is direct evidence that quality of enrichment matters, but the variables were hand-coded at a single center.
On the extraction side, Robitschek et al. supplies the method. It used GPT-4-turbo to pull 23 social-determinant dimensions from 4,331 transplant notes at 85.9% average accuracy. It also constrains the design in two ways: it was validated only within UCSF, and it acknowledges the documentation "streetlight effect." We reuse its prompt-and-verify style. We differ by targeting reserve constructs, validating extraction across institutions, and treating "not documented" as an explicit, modeled outcome. CLEAR supplies the idea of decomposing each attribute into explicit present, absent, and unknown slots. This matters because its gains over rule-based labelers were concentrated on negations, which is exactly where "no occupation recorded" and "never worked" must be told apart.
The brain side is shaped by a negative result. Mind the Gap shows that a scalar brain age gap classifies Alzheimer's disease poorly (AUC about 0.70–0.71) compared with direct 3D CNNs or fine-tuned intermediate features (about 0.92–0.93). We take this seriously and do not use the scalar gap as our damage measure. Instead we use regional residuals from coVariance neural networks, which are graph networks over a covariance matrix of cortical regions. Sihag et al. showed that these separate diseases with overlapping scalar gaps, and the graph-signal-processing follow-up showed they transfer across atlas resolutions (MAE 5.39 to 5.61 years). Our contribution is not a better brain age model. It is the use of an existing one as a yardstick for reserve.
Research Program
The imaging arm trains a coVariance network on healthy OASIS-3 cortical thickness, following the published recipe. It then computes regional residuals on ADNI, AIBL, MACC, and two partner memory clinics with routine MRI. A small site-adversarial or compositional-alignment head, in the spirit of Compositional Feature Alignment, is an optional step. We include it only if site identity remains decodable from the residuals after the held-out-site fit.
The text arm requires partner clinics, because the public cohorts in our file do not release clinical notes. We will work with two memory clinics in different health systems, one Western and one Southeast Asian. This mirrors the geographic skew documented by Vipin et al. At each clinic we extract occupational complexity, literacy or reading habits, bilingualism, educational quality, social engagement, and cognitively demanding leisure. Each attribute is coded as present, absent, or not documented, with the supporting sentence quoted.
Extraction is validated in two ways. The first is clinician annotation of a few hundred notes per site. The second is larger in scale: at clinics that also administer a structured reserve or lifestyle questionnaire, the questionnaire serves as ground truth for thousands of patients. This lets us measure cross-site extraction transfer properly, rather than on about 100 notes.
The analysis arm fits $\hat f$ with leave-one-site-out cross-fitting, computes $\rho$, and runs the tests below.
Baselines
The naive reference point is reserve proxied by structured years of education, entered directly into a model of cognition alongside chronological age. We also test an even simpler variant with no reserve term at all, so that $\rho$ is just cognition adjusted for age. Every richer model must beat this on held-out sites. On the same patients and folds, it must explain more variance in $\rho$ and in longitudinal decline slopes. We will report the result with site-clustered bootstrap intervals that exclude zero.
The strongest existing extraction method is the GPT-4-class prompt-based pipeline of Robitschek et al. It reports 85.9% average accuracy over 23 dimensions on 101 single-site notes. We run that recipe unchanged on our reserve attributes as the extraction baseline. Our version is the same model with slot decomposition, an explicit "not documented" slot, and sentence grounding. It counts as better if it reaches higher accuracy at the external site with equal prompt-tuning effort. Prompts are frozen after tuning at site A only, and both versions use the same model and token budget.
The strongest brain-side baseline is a direct supervised MRI-to-cognition predictor, the 3D CNN family that reaches 0.92–0.93 AUC in Mind the Gap. Its residual serves as an alternative "reserve" score. Mind the Gap also reports the scalar gap baseline at 0.70–0.71 AUC for Alzheimer's classification. For the specific task of predicting a reserve residual or reserve-driven decline, we found no reported result on Lacuna, so all residual comparisons are new.
Three ablations isolate the key ingredients. The first replaces regional residuals with a scalar gap. The second fits $\hat f$ by pooled cross-validation instead of whole-site holdout. The third replaces LLM extraction with keyword rules. If any of these matches the full method on the decisive checks, that ingredient is unnecessary and we will say so.
Why not just…?
Why not just use years of education? This is the real threat, and we treat it as the null hypothesis rather than a strawman. Petersen et al. found reading quality mattered where schooling did not, and Vipin et al. show that schooling has paradoxical longitudinal effects. Both point to schooling being an incomplete proxy, but neither proves notes add anything. The incremental-variance test on held-out sites settles the question. If extracted features add nothing over education, the proposal's reportable result becomes the cross-site residual protocol alone, and we will report the extraction arm as negative.
Why not just predict cognition directly from MRI instead of detouring through brain age? Direct predictors win at diagnosis, as Mind the Gap shows. But a model trained to predict cognition learns to explain away reserve: a resilient patient's MRI-to-cognition residual shrinks precisely because the model has fitted the patterns that accompany preserved function. A cognition-blind normative model cannot do this. We test the argument empirically. If the direct predictor's residual is as strongly predicted by enrichment history and reproduces the education paradox equally well, the normative detour is unneeded.
Why not just use structured codes or keyword rules? Robitschek et al. show that psychosocial factors largely live in free text. CLEAR shows that rule-based labelers fail most on negations, by up to 42.5 points. Our keyword ablation measures whether this holds for reserve attributes, which may be more often stated plainly ("retired engineer") than social risk factors are. If it does not hold, rules are the better choice.
Why not just pool all sites and harmonize? Pooled fitting lets site-specific scanner offsets leak into $\hat f$, so residuals partly encode site. Adaptive-CaRe (Adaptive-CaRe) shows how pooled ERM can rely on shortcuts: 99% train accuracy fell to about 76% under shifted shortcuts. The pooled-versus-held-out ablation, scored by how well site can be decoded from $\rho$, answers this directly.
Decisive Checks
Four checks decide whether the residual is a reserve score. The first is a site-leakage check: a classifier must not recover site or scanner from $\rho$ above chance. If it can, the residual fails regardless of other results.
The second is the incremental-value check against education on held-out sites, described in Baselines.
The third is the paradox signature. In the longitudinal subsets of ADNI and the partner clinics, high-$\rho$ patients should decline more slowly before diagnosis and faster after it. A residual lacking this crossover is suspect. A similar pattern around acute insults would also fit the finite-reserve "vulnerability window" picture of Guan et al., but we treat that as secondary.
The fourth is a documentation-bias check. Extracted features must keep their predictive value for $\rho$ after conditioning on note count, note length, and insurance or area-deprivation proxies. A further version restricts the analysis to patients whose attributes are all "documented."
If the first and fourth checks pass but the second fails, we have a portable reserve measure and evidence that clinical notes add little beyond schooling. That is still a useful answer. If all four pass, notes become a scalable source of reserve measurement for health systems that never administered a reserve questionnaire.
References
- Dementia in Southeast Asia: influence of onset-type, education, and cerebrovascular disease
- Multimodal Brain Age Gap as a Mediating Indicator in the Relation between Modifiable Dementia Risk Factors and Cognitive Functioning
- Effects of clinical, comorbid, and social determinants of health on brain ageing in people with and without HIV: a retrospective case-control study
- A large language model-based approach to quantifying the effects of social determinants in liver transplant decisions
- CLEAR: A Clinically Grounded Tabular Framework for Radiology Report Evaluation
- Mind the Gap: Does Brain Age Improve Alzheimer’s Disease Prediction?
- Explainable Brain Age Gap Prediction in Neurodegenerative Conditions Using Covariance Neural Networks
- Disentangling Neurodegeneration With Brain Age Gap Prediction Models: A graph signal processing perspective
- Enhancing Compositional Generalization via Compositional Feature Alignment
- Adaptive-CaRe: Adaptive Causal Regularization for Robust Outcome Prediction
- Changes in a sensorimotor network, occipital network, and psychomotor speed within three months after focal surgical injury in pediatric patients with intracranial space-occupying lesions