Doctors judge cognitive risk from small-vessel brain disease in two ways. A neuropsychologist can give a one-on-one test such as the MoCA or the Symbol Digit Modalities Test (SDMT), or a neurologist can read a FLAIR MRI and score how much white-matter damage sits on the brain's cholinergic pathways. FLAIR is a routine MRI sequence in which these lesions, called white matter hyperintensities (WMH), show up bright. The pathway reading is the Cholinergic Pathways Hyperintensities Scale (CHIPS). Both routes are slow and depend on the person doing them. Junior residents scoring CHIPS on the same scans agree poorly (Kendall's W = 0.555), and one automated pipeline cut scoring time from over 123 seconds to 4.3 seconds per patient (Fei et al.). That pipeline segments lesions slice by slice with a 2D U-Net and then applies a fixed atlas threshold. The authors name the move to 3D as future work.
We propose that the next step is not a 3D version of the same score. We want a 3D network whose whole-brain features are pooled under a soft weight map, built from the predicted lesion probabilities and the cholinergic pathway atlas, and trained to predict cognitive test scores directly. The main question is whether this learned representation predicts cognition better than the automated CHIPS score computed from the same 3D segmentation. The CHIPS-plus-linear-model route is the obvious, cheap competitor, and the project is only worth doing if we can beat it.
Core Claim
Hand-crafted pathway scores keep two pieces of information: where lesions sit and how large they are. They discard lesion shape and connectivity across slices, and they discard the state of the normal-appearing white matter just around the lesions. Whole-brain 3D CNNs keep all of that. However, they spend their capacity on everything else in the scan, including signatures of the acquisition site. Unmasked whole-brain 3D CNNs reach 94.1% accuracy for Alzheimer's disease versus normal controls but only 61.1% across three classes (Esmaeilzadeh et al.). On early mild cognitive impairment versus controls, the AUC falls to 0.57–0.58 (Korolev et al.).
We claim that a third option outperforms both on two measures: predicting continuous cognitive scores, and holding up on unseen scanners. In this option the network sees the whole volume and is trained end-to-end, but its final pooling is weighted toward pathway-registered lesions and a learnable band of tissue around them. The claim is falsifiable in a specific way. If the pooled features add no predictive value over 3D-automated CHIPS plus total WMH volume, then the field should keep the interpretable score and stop.
Research Program
The system has three parts.
Part 1: lesion segmenter. This is a 3D WMH segmenter trained on the MICCAI WMH Challenge data (Kuijf et al.). It borrows two techniques from earlier work. The first is a white-matter atlas prior from the brain-atlas-guided attention U-Net (BAGAU-Net), which lets FLAIR-only models match FLAIR+T1 models. The second is MixDANN's domain-adversarial training with mixup (Zhao et al.). Both were shown in 2D, so we lift them to 3D.
Part 2: soft weight map. Each scan is registered to MNI space. For each voxel $v$ and cholinergic region $r$, we set the weight
$$w_r(v) = a_r(v)\,\big[\,p(v) + \lambda\, s_\tau(v)\,\big],$$
where the terms are as follows. $a_r(v)$ is the probabilistic atlas membership of voxel $v$ in pathway region $r$, the same 20-region parcellation CHIPS uses. $p(v)$ is the segmenter's lesion probability, not a thresholded mask. $s_\tau(v)$ is a soft shell around lesions, obtained by Gaussian dilation of $p$ with width $\tau$. $\lambda$ sets how much perilesional tissue counts. Both $\lambda$ and $\tau$ are learned.
Part 3: cognitive predictor. A shallow 3D CNN encodes the full FLAIR volume. It is kept around $10^4$–$10^5$ parameters because deeper 3D networks overfit cohorts of a few hundred (Esmaeilzadeh et al.; Korolev et al.). Its feature map is pooled once per region under $w_r$. This extends the mask-weighted global average pooling used for lung CT (Lee et al.), where a smoothed lung mask raised specificity by 2% over plain pooling. We swap the single organ mask for twenty pathway-specific, lesion-dependent weights. The region vectors, together with the CHIPS score computed from the same weights, feed a regression head. That head predicts MoCA, MMSE and SDMT, plus a binary flag for moderate-to-severe impairment.
Why gradients reach the whole brain. The encoder sees the whole brain, so receptive fields carry context from far away. Only the read-out is anatomically constrained. This is how we respond to the argument that pre-selecting a region of interest throws away global context (Esmaeilzadeh et al.).
What we analyze. We fit the model and also analyze it. We measure how much a site classifier can recover from the pooled features compared with unpooled features. We check whether the learned $\lambda$ is reliably nonzero, which would be evidence that perilesional tissue carries cognitive signal. We also measure how 2D versus 3D segmentation errors spread into CHIPS error. That last analysis matters because the WMH challenge FLAIR is acquired as 2D multi-slice, with thick slices. In that setting the advantage of 3D processing may shrink, and we will report that honestly rather than assume it.
Baselines
Naive reference point. A linear model on age, sex and total WMH volume from the 3D segmenter. Any method that does not beat this is not using spatial information at all.
Strongest clinical competitor. A reimplementation of Fei et al., run in two versions: as published, with the 2D U-Net and atlas thresholding, and with our 3D segmenter swapped in. Each version's CHIPS score goes into a linear model for cognition. The 3D-CHIPS version is the baseline that matters most, because it is exactly what Fei et al. propose as future work. On their own cohort, Fei et al. report correlation with senior neurologists of r = 0.941 internally and 0.846 externally, and AUC of 0.903 internally and 0.868 externally for moderate-to-severe cholinergic injury. We will compare against our reimplementation on shared data, not against those numbers. Beating this baseline means a statistically reliable gain in cognitive-score correlation, or in impairment AUC, on held-out sites, judged by bootstrap confidence intervals.
Unconstrained whole-volume model. VoxCNN (Korolev et al.) on skull-stripped FLAIR. We also run the 8-layer 3D-CNN with bounding-box cropping from Wang et al.. That paper reports 96.9% four-class accuracy on its private 263-subject cohort, against about 70% for 2D VGG-16. We treat that figure as unverified and do not use it as a target, because the cohort is small, comes from one centre, and has no external test.
Compression approach. For the Alzheimer's staging transfer check described below, we include MVSC (Multimodal Visual Surrogate Compression). It reports a three-class macro-AUC of 93.65% on ADNI and 83.01% on OASIS-3.
Ablations of the key ingredient. Each removes one piece of the weight map. The first uses a hard thresholded mask instead of $p$. The second uses the atlas only ($p$ replaced by 1). The third uses lesions only ($a_r$ replaced by 1). The fourth fixes $\lambda = 0$, so there is no perilesional shell. The fifth uses plain global average pooling.
Fairness controls. Every model gets the same splits, the same registration, the same preprocessing, and an equal hyperparameter search budget: 30 trials, selected on a validation fold from the training sites only.
Why not just…?
Why not just run Fei et al.'s pipeline with a 3D U-Net? This is the real threat, and it may win. CHIPS already correlates with MMSE, MoCA and SDMT at p < 0.0001 (Fei et al.), and it is interpretable. Our design includes the CHIPS score as an input to the head, so the learned features have to earn their place as residual signal. The atlas-only and $\lambda = 0$ ablations show where any gain comes from. If 3D-CHIPS plus a linear model ties us, the right conclusion is to deploy the simpler system. That would still be a useful result, since it would settle the 2D-versus-3D question for CHIPS automation.
Why not just use an unmasked whole-brain 3D CNN? Unmasked 3D CNNs are near chance on early-stage distinctions (Korolev et al.). On 10,420 multi-site scans they also stay biased toward site signatures even with augmentation (Dufumier et al.). Our leave-one-site-out evaluation and the site-classifier probe test directly whether restricting the read-out removes that bias. If the unmasked model generalizes equally well, the pooling adds nothing.
Why not just use voxel-based morphometry with a ridge model? Dufumier et al. found that in cohorts under 1,000 scans, linear models on voxel-based morphometry match CNNs. Our cohorts are in that range. We include ridge regression on modulated white-matter maps, plus lesion-probability maps in MNI space, as a baseline. This is a strong competitor, and losing to it would argue against deep pooling altogether.
Proposed Experiments
Data.
- Segmentation. We train on the WMH Challenge and test leave-one-scanner-out. For context, MixDANN reports a Dice score of 0.694 on the unseen Utrecht domain when trained on Amsterdam and Singapore, against 0.183 for a plain U-Net. BAGAU-Net reports 82.02% Dice.
- Cognition, public data. This needs subjects with FLAIR and cognitive scores across scanners. We plan to use ADNI's FLAIR-bearing subset, although the file documents only T1 use and FLAIR availability must be confirmed.
- Cognition, clinical cohort. Where access allows, we will use the silent cerebrovascular disease CHIPS cohort from Fei et al. This has 464 internal patients and 100 external patients across GE and Philips scanners at 1.5T and 3.0T, and it lets us compare like for like.
- Transfer check. We apply the same pooling to T1 hippocampal and ventricular atlas regions for three-class Alzheimer's staging on ADNI, AIBL and OASIS-3. For this setting, no reported result on Lacuna was found for a mask-weighted 3D model.
Metrics. For cognition: Pearson and Spearman correlation and mean absolute error against MoCA and SDMT, and AUC for impairment. We also report the gap between internal and external sites, and site-probe accuracy.
Decisive checks. There are three.
- Learned features versus the score. Does our model beat 3D-CHIPS plus linear regression on held-out scanners, with confidence intervals that exclude zero?
- Perilesional tissue. Is the learned $\lambda$ above zero, and does ablating the shell hurt?
- Scanner robustness. Does pooling make the internal-to-external drop smaller than the 0.941 to 0.846 decline Fei et al. observed?
If the first check fails, we report the 3D-CHIPS system as the recommended tool.
References
- Intelligent cholinergic white matter pathways algorithm based on U-net reflects cognitive impairment in patients with silent cerebrovascular disease
- End-To-End Alzheimer's Disease Diagnosis and Biomarker Identification
- Residual and plain convolutional neural networks for 3D brain MRI classification
- Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge
- Brain Atlas Guided Attention U-Net for White Matter Hyperintensity Segmentation
- Robust White Matter Hyperintensity Segmentation On Unseen Domain
- Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling
- Classification of Subcortical Vascular Cognitive Impairment Using Single MRI Sequence and Deep Learning Convolutional Neural Networks
- Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification
- Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning