Memory clinics do not agree on how to measure cognition. One site gives the Mini-Mental State Examination (MMSE), another gives Addenbrooke's Cognitive Examination-III (ACE-III), and a third records speech tasks such as naming, fluency and story recall. Every one of these batteries is really measuring the same handful of underlying abilities: memory, executive function and processing speed, language, attention, and visuospatial skill. Clinical work shows that these abilities matter separately. Executive slowing appears before memory loss in amyloid-positive people who are still cognitively normal, and genetic risk shifts some abilities up while pushing others down. Machine learning for dementia has mostly ignored this. Models either predict a single diagnosis label, or they convert every battery into one scalar score with a lookup table, and they pool raw speech or brain scans on a single machine, which privacy law often forbids.
We propose to train one model across clinics without moving their data. The model maps a patient's speech or brain scan to a small vector of cognitive-domain scores. Each clinic keeps a private "measurement layer" that translates that vector into whatever subscores its own battery produces. The approach borrows the logic of factor analysis: tests load on a few latent abilities. A handful of tests that several batteries share, such as delayed word recall and animal fluency, act as anchors, so that "memory" means the same axis at every site. The result is a cognitive profile that can be estimated for any patient at any participating clinic, including for domains that the clinic's own battery never tested, learned from data that never left the clinic.
Core Claim
Disagreement between batteries is usually treated as a nuisance, something to be converted away before training. We argue it should instead be modelled as a measurement problem inside the federated objective. Let $z(x) \in \mathbb{R}^K$ be the domain vector that a shared encoder produces from input $x$ (speech or MRI), with $K$ around five. Each test $j$ is then modelled as a noisy linear readout of that vector:
$$ \hat{y}_{j}(x) = \lambda_j^{\top} z(x) + b_j , $$
where $\lambda_j \in \mathbb{R}^K$ holds the loadings of test $j$ on the domains and $b_j$ is its offset. Training minimises squared error only on the tests a patient actually took. Anchor tests, those present at two or more sites, have loadings that are shared and aggregated by the server, with a sparsity pattern fixed in advance (recall loads on memory, fluency loads on executive function and language). Tests specific to one site have loadings that stay on that site and are never transmitted. This follows the partially local design of Federated Reconstruction, which splits parameters into global and never-shared local parts and rebuilds the local part with a few gradient steps. Three ingredients are new here. The local part is a structured, low-rank measurement model rather than a free prediction head. The anchors make the latent axes identifiable and comparable across sites. And the evaluation asks whether the model can predict domains a site never measured.
Relation to Prior Work
The closest federated work on dementia motivates the proposal by what it leaves out. UG-FedDA federates structural MRI across ADNI, AIBL and OASIS. It uses uncertainty-weighted domain alignment and an evidential classifier for normal control, mild cognitive impairment (MCI) and Alzheimer's disease (AD), and it down-weights borderline MCI cases as ambiguous. ClusMFL handles clinics that lack some imaging modalities by using cluster centres as stand-ins for the missing ones, but it assumes every site shares the same three-class label set. Both papers solve the problem of incomplete or shifted inputs. We solve the mirror problem of incomplete and heterogeneous labels. We also predict continuous profiles rather than discrete stages.
On the clinical side, Susanto et al. show three things in ADNI: executive function and processing speed (EF/PS) decline in amyloid-positive cognitively normal people before memory does; carriers of the APOE ε4 gene variant show worse memory but better EF/PS; and a classifier built from EF/PS, parietal grey-matter volume and APOE reaches AUC 0.781 for detecting abnormal cerebrospinal-fluid amyloid. Dong et al. show that a stroke screen without executive and visuospatial items predicts one-year impairment worse than the full MoCA (AUC 0.80 versus 0.89). Together these papers justify profiles over scalars. They also give us biological checks that a learned profile should pass.
Automatic cognitive assessment: Combining sparse datasets with disparate cognitive scores is the closest work on mixing batteries. It pools speech data scored with MMSE (ADReSSo) and with ACE-III (IVA) on one machine, and it converts ACE-III to MMSE with a published crosswalk. Our proposal departs from it twice: we keep the ACE-III subdomains rather than collapsing them, and we never pool the data. Federated Multi-Task Learning (MOCHA) treats each client as a task and learns how clients relate. In our setting the tasks are cognitive domains that cut across clients, so MOCHA's formulation does not apply directly. Its convex-only guarantees also limit it to shallow models. FedHealth personalises upper layers for activity recognition and supplies the general recipe of a shared body with local personalisation that we start from. Privacy-first health research with federated learning shows that federated regression can reproduce centralised clinical estimates; one example is a SARS-CoV-2 odds ratio of 0.35 in both settings. This supports the claim that federation need not cost accuracy, but every endpoint in that work is a single scalar.
Research Program
We build two testbeds.
The first is controlled and uses ADNI imaging, where domain-level neuropsychological scores exist for memory, EF/PS, language, visuospatial ability and attention. We split ADNI into simulated clinics and give each clinic only a subset of the tests, mimicking different batteries. Every clinic keeps at least one memory and one executive anchor, and some clinics have no visuospatial or language tests at all. Because the withheld scores exist, we can measure directly whether the model recovers domains a clinic never measured. We then add AIBL and OASIS as real external sites with their own scanners and assessments, following the cohort setup of UG-FedDA.
The second testbed is speech. The PARLO Dementia Corpus supplies nine German memory clinics as natural federated sites. Each recording comes with task-level speech scores (naming, semantic fluency, repetition, delayed recall) and MMSE. ADReSSo supplies an English site with MMSE only. IVA, scored with ACE-III subdomains, is included only if its holders join as a federated node, which is exactly the scenario this method exists to serve. The speech encoder works on acoustic features together with local speech-recognition transcripts. Transcription runs on site, and the model receives the recogniser's confidence as an input. This matters because PARLO reports Whisper word error rising from 9.0% to 15.2% in dementia patients, so transcription quality itself carries a cognitive signal.
The analyses cover three questions. The first is accuracy on observed subscores. The second is accuracy on withheld domains. The third is whether the latent axes are scientifically meaningful. On that last point, the learned EF/PS axis should separate amyloid-positive from amyloid-negative cognitively normal people better than the memory axis does, and APOE ε4 carriers should score lower on the memory axis and higher on EF/PS. Both predictions come from Susanto et al. A model that merely fits subscores well, but whose axes are rotated mixtures, will fail this test. Training also uses the stateless reconstruction trick from FedRecon: a site's local loadings are rebuilt from a support set each round, so a new clinic can join with a few gradient steps. Differential privacy on the shared updates is an optional arm, used to measure the cost to utility.
Baselines
The naive reference points are local-only training and isolated scalar federation. In the local-only baseline, each clinic fits the same encoder and measurement model on its own patients. PARLO averages about 23 patients per clinic, so we expect this to overfit badly, and it sets the floor. The scalar federation baseline follows the protocol of Automatic cognitive assessment: convert every battery to MMSE with the crosswalk where one exists, then train a single regression head with FedAvg. The strongest reported numbers for this kind of pipeline are an MMSE RMSE of 5.28 for the ADReSSo challenge baseline (ADReSSo), and 4.45 standalone versus 4.40 pooled with IVA (not significant), plus 3.00 on IVA when pooled with ADReSSo, all in a centralised setting (Automatic cognitive assessment). Matching roughly 4.4 RMSE on the ADReSSo test set without pooling, while also producing domain profiles, would be the bar for MMSE prediction.
The strongest simple competitor is a masked multi-head FedAvg model. It uses the same encoder and one free linear head per test. All heads are shared and averaged, and the loss ignores tests a patient did not take. This model can also predict missing tests at any site, so it is the real threat to our method. A FedRecon-style variant keeps those free heads local. The key ablations remove each ingredient in turn: our model without the anchors, our model without the rank-$K$ bottleneck (equivalent to the free-head baseline), and our model without fixed sparsity patterns.
We also run a centralised pooled version of our model as an upper bound, not as a competitor. On imaging we report UG-FedDA's three-class accuracies for comparison: 90.54% for normal versus AD on ADNI, 89.04% on AIBL and 77.78% on OASIS. We obtain these by thresholding our profiles with a small classifier, which tests whether moving to profiles costs anything on staging. For federated prediction of a multi-domain vector, we found no reported result on Lacuna.
All methods share the same encoder, the same input preprocessing, the same number of communication rounds and local steps, and the same hyperparameter search budget per method, tuned on site-held-out validation splits. On the ADNI testbed we repeat runs over at least five random assignments of tests to sites. Beating the free-head baseline means two things: lower error on withheld domains, with confidence intervals that do not overlap across the battery assignments, and passing both biological checks where the baseline does not.
Why not just…?
Why not just convert every battery to MMSE and federate a scalar? This discards exactly the signal the clinical literature says matters. The 5-minute stroke protocol lost AUC (0.80 against 0.89 for the full MoCA) precisely because it lacked executive items (Dong et al.). The crosswalk approach also gave no significant gain on ADReSSo (4.45 to 4.40). The scalar baseline is in our experiments, and the amyloid-positive cognitively normal detection test shows directly whether a scalar can see what an EF/PS axis sees.
Why not just share a free head per test with masked FedAvg? This is a genuine threat. On observed subscores it may match or beat us, because it has more freedom. Our bet is that this freedom hurts transfer: with tests sparsely spread across sites, free heads can fit spurious, site-specific directions, and nothing forces their latent space to mean the same thing everywhere. The ablation without the bottleneck settles the question. If free heads match us on withheld domains and on the APOE and amyloid checks, we will report that the structure is unnecessary and that the contribution reduces to a federated multi-battery benchmark.
Why not just pool de-identified data centrally? Raw voice recordings and 3D MRI are biometric and re-identifiable (Privacy Policy and Technology in Biomedical Data Science), and the nine-clinic PARLO setting is exactly the kind of multi-site arrangement where pooling is hard to obtain. We still run the pooled model as an upper bound, so the price of federation is measured rather than assumed.
Why not just use MOCHA, which already does federated multi-task learning? MOCHA's tasks are clients, and its guarantees need convex models. Here the tasks are domains shared across clients, and the encoder is deep. It could serve as a shallow baseline on fixed features, and we include it in that form on the ADNI testbed.
Decisive Checks
The proposal succeeds if three things hold together. First, on the ADNI testbed, the anchored model predicts withheld domain scores at each simulated clinic better than the masked free-head baseline. Second, its learned axes reproduce the EF/PS-before-memory signal in amyloid-positive cognitively normal participants, roughly matching the 0.781 AUC of Susanto et al.'s handcrafted classifier, and the APOE ε4 dissociation between memory and EF/PS. Third, on PARLO's nine clinics, federated training beats local-only training on task-level scores and stays close to the pooled upper bound. The idea fails if the anchors add nothing over free heads, or if the axes fail the biological checks while still fitting subscores well. That outcome would show that subscore fit alone is not evidence of a meaningful cognitive profile, which is itself worth reporting.
References
- Federated Reconstruction: Partially Local Federated Learning
- UG-FedDA: Uncertainty-Guided Federated Domain Adaptation for Multi-Center Alzheimer's Disease Detection
- ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis
- Cognition, Brain Atrophy, and Cerebrospinal Fluid Biomarkers Changes from Preclinical to Dementia Stage of Alzheimer's Disease and the Influence of Apolipoprotein E
- The Montreal cognitive assessment is superior to national institute of neurological disease and stroke-Canadian stroke network 5-minute protocol in predicting vascular cognitive impairment at 1 year
- Automatic cognitive assessment: Combining sparse datasets with disparate cognitive scores
- Federated Multi-Task Learning
- FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
- Privacy-first health research with federated learning
- The PARLO Dementia Corpus: A German Multi-Center Resource for Alzheimer's Disease
- Detecting Cognitive Decline Using Speech Only: The ADReSSo Challenge
- Privacy Policy and Technology in Biomedical Data Science