In multiple sclerosis, small vessel disease and early Alzheimer's disease, brain tissue is damaged long before anything shows up on a clinical scan. Radiologists cannot label damage they cannot see, so no one can train a segmentation network for it in the usual way. Specialized diffusion MRI can measure this damage. Diffusion MRI tracks how water moves in tissue, and "multi-compartment" models split each voxel's signal into water inside axons, water around them, and free fluid. These scans need long acquisitions and strong hardware, so they almost never appear in routine care. We propose to use the biophysical model as the annotator. In a research cohort scanned with both protocols, the multi-compartment maps would supply dense, voxel-level training targets with no human labeling. A network trained on those targets would then predict the same deviation maps from ordinary T1, T2 and FLAIR scans, which hospitals already collect.
The opening is supported by two findings in the file. The first establishes that the target exists. Diffusion models called NODDI and the spherical mean technique (SMT) find axon-density loss across about 12% of normal-appearing white matter in relapsing-remitting multiple sclerosis, and that loss tracks disability (Comparison of NODDI and SMT Parameter Maps in MS). The second suggests the input carries enough information. A linear classifier on plain 1.5T T2 scans separated patients from controls with 91% accuracy using only normal-appearing white matter (MRI Pattern Recognition in MS Normal-Appearing Brain Areas). What is missing is a model that turns that diffuse, group-level structural signal into a spatial map with physical meaning, without anyone drawing a mask.
Core Claim
A network trained on routine structural MRI can predict voxelwise deviation from healthy norms in the intra-axonal fraction and the free-water fraction. The claim covers normal-appearing white matter away from visible lesions. Accuracy should be good enough to reproduce the clinical associations that the real diffusion maps show, and it should beat structural-only alternatives that skip the biophysical target. The training targets come from physics rather than experts. Using two imperfect biophysical teachers, NODDI and SMT, and weighting voxels by how much they agree should matter, because the two models agree only partly. Their axon-density estimates correlate at r = 0.72, and the regions where each finds significant patient–control differences overlap with Dice 0.42 (Johnson et al.).
Relation to Prior Work
The closest work does weakly supervised white matter segmentation, but every method aims at visible lesions. WMH-DualTasker trains on radiologists' visual rating scores to segment FLAIR hyperintensities. It explicitly treats all non-hyperintense tissue as healthy background, which is exactly the tissue we target. It also supplies our downstream reference numbers. Fusing Unsupervised and Supervised Deep Learning contributes the method we reuse in spirit: a student network trained on noisy machine-generated labels outperformed its teacher, a healthy-trained autoencoder. That paper's teacher was an intensity residual on the same modality, thresholded into binary masks. Its documented false positives at ventricles and sulci constrain our design: we do not take our teacher from structural-image residuals. PathoGAN segments tumors from healthy-versus-diseased labels alone. Tumors are large, mass-effect lesions, and the method's binary formulation cannot represent graded decay.
Two further papers motivate the problem without solving it. The searchlight classifier of Weygandt et al. shows that sub-visual information exists in T2. It produces only group classification, and it needed manual lesion tracing first. The tauopathy study Beyond Neurofibrillary Tangles found that classifiers base their decisions on diffuse background tissue rather than focal tangles. That finding also motivates the work, and it also warns us: saliency maps from a diagnosis classifier are interpretations, not measurements. Finally, SpurCon shows how readily medical classifiers learn shortcuts: worst-group accuracy was 0% on ISIC 2020 before correction. Scanner signatures are the obvious shortcut for sub-visual texture, so we test for them explicitly.
Research Program
The teacher maps come from a paired cohort scanned with multi-shell diffusion and the standard T1, T2 and FLAIR protocol, following the design of the Johnson et al. cohort of 63 patients and 28 controls at 3T. We fit NODDI and SMT in every subject. Each control's maps are registered to a common space, and we build a voxelwise normative model of each parameter as a function of age and sex. For subject $s$, voxel $x$ and parameter $m$ (intra-axonal fraction or free-water fraction), the target is the deviation
$$z_{s,m}(x) = \frac{m_s(x) - \mu_m(x \mid \text{age}_s, \text{sex}_s)}{\sigma_m(x)},$$
where $\mu_m$ and $\sigma_m$ are the normative mean and spread estimated from controls. A 3D U-Net receives only the structural images and predicts $\hat z_{s,m}(x)$. Training uses a regression loss in which each voxel is weighted by NODDI–SMT agreement. The weight $w(x)$ is high where the two teachers' deviations have the same sign and similar size. It is near zero where they conflict, and near zero within a few millimetres of the cerebrospinal fluid boundary, where partial-volume effects corrupt free-water estimates. Controls enter training as well, so the network learns that healthy tissue maps to near-zero deviation.
The analysis tests three things. The first is whether the predicted maps match the held-out teacher maps in normal-appearing white matter. The second is whether they carry the teachers' clinical signal. The third is whether they survive a change of scanner. The output is a pair of continuous maps that can be computed on any routine clinical scan. One map estimates axonal loss and the other estimates excess free fluid.
Baselines
The do-nothing reference is the healthy-norm prediction $\hat z = 0$ everywhere, together with an age-and-lesion-load-only predictor. The second is a linear model that assigns each voxel a deviation from the subject's age, total FLAIR lesion volume, and distance to the nearest visible lesion. It tests whether the network merely learns "tissue near lesions and in older brains is worse." Both are evaluated in normal-appearing white matter at least 3 mm from any hyperintensity. Our model counts as an advance only if it clearly beats the distance-and-load predictor there.
The strongest structural-only competitor is a voxelwise normative z-score of the structural intensities themselves. T2 and FLAIR values are intensity-normalized and compared against the same controls, then mapped to diffusion deviation by a per-region linear fit. This is the most direct deep-learning-free reading of the Weygandt et al. result. It gets the same training subjects and the same normal-appearing-white-matter mask.
The second competitor is the Baur et al. pipeline: a healthy-trained spatial autoencoder followed by a student U-Net trained on its residuals. It is retrained on our structural data with the same compute budget and hyperparameter search. In its original setting it reports Dice 0.6343 for the autoencoder and 0.6792 for the student on MS lesions (Baur et al.). Because the task differs, we compare it on our continuous targets rather than on those Dice numbers.
The third competitor is a classifier trained on patient versus control, whose class-activation maps are read as injury maps. It represents the weak-label route. There is no reported result on Lacuna for predicting NODDI or SMT deviation from structural MRI, so all of these baselines run on our data under matched architecture size, number of training epochs and tuning trials.
Two ablations isolate the key ingredients. The first replaces the two-teacher agreement-weighted target with NODDI alone. The second replaces the multi-compartment target with ordinary single-tensor DTI deviation, the fractional anisotropy and mean diffusivity of the standard diffusion tensor model. If the DTI-trained student is just as clinically useful, the biophysical decomposition is not earning its cost.
Why not just…?
Why not just z-score FLAIR and T2 intensities against healthy controls? This is the most serious threat, because the 91% classification result says the structural signal is already present. Intensity z-scores, however, mix free fluid, myelin and iron, and the file's direction summary notes that opposing changes in a voxel can cancel in an averaged signal. We expect the intensity baseline to track free-water deviation reasonably well and axonal-fraction deviation poorly. If it matches our model on both parameters in lesion-distant white matter, we will report that the learned model adds nothing beyond normative intensity mapping.
Why not just use a healthy-trained autoencoder, as in prior unsupervised lesion work? Baur et al. document false positives at high-contrast boundaries. More fundamentally, an autoencoder trained to reconstruct healthy anatomy will reconstruct a slight, diffuse shift as healthy, so its residual vanishes exactly where our target lives. The matched comparison above settles this. We will also report correlation with teacher maps in periventricular versus deep white matter separately, to see whether any autoencoder advantage is a boundary artifact.
Why not just train a diagnosis classifier and read its saliency maps? Class-activation maps are coarse. The WMH Segmentation Challenge found that every method failed on small lesions. The tauopathy study shows that classifiers attend to diffuse tissue for reasons no one specified. A saliency map has no physical units and cannot be checked voxel by voxel, whereas our target can. The saliency baseline is scored against the same teacher maps, so this objection gets a direct answer.
Why not just acquire multi-shell diffusion clinically? Where that is possible, the real maps should be used. Our model matters for the large existing archives and routine protocols that lack the scan. We do not claim the student can replace the teacher; the claim is only that it recovers enough of the teacher's signal to be useful where the teacher is absent.
Proposed Experiments
The first decisive check uses subject-level cross-validation in the paired cohort. We report the voxelwise correlation and mean absolute error between predicted and teacher deviation in lesion-distant normal-appearing white matter. The model must beat the distance-and-load predictor and the intensity z-score baseline in both parameters.
The second check asks whether the clinical signal survives. We regress disability (EDSS) on each subject's mean predicted axonal-fraction deviation in normal-appearing white matter. The NODDI teacher itself reports a standardized beta of −0.34 there (Johnson et al.). A student whose beta keeps the same sign, has comparable magnitude, and has an overlapping confidence interval, while the baselines' betas are weaker, would show that the clinical meaning transfers.
The third check targets scanner shortcuts and uses the WMH Segmentation Challenge data: 170 scans from five scanners and three vendors with T1 and FLAIR. On this data, a classifier asked to identify the scanner from predicted normal-appearing white matter maps should do far worse than one given raw intensity features, after adjusting for lesion load and age. If scanner identity is easy to read from our maps, they are encoding hardware, and the method fails on its own terms.
The fourth check tests downstream value on clinical-only data in ADNI, where routine structural scans exist. We add predicted normal-appearing white matter deviation to lesion volume for classifying mild cognitive impairment. The reference is the AUC of 0.718 that WMH-DualTasker reports for that task, with 0.652 for conversion to dementia. We will recompute WMH-DualTasker's features on our exact subject split before claiming a gain. A gain that holds across folds would be the first evidence that physics-supervised maps of invisible injury add diagnostic information beyond what a radiologist's visible lesions provide.
References
- Comparison of Neurite Orientation Dispersion and Density Imaging and Two-Compartment Spherical Mean Technique Parameter Maps in Multiple Sclerosis
- MRI Pattern Recognition in Multiple Sclerosis Normal-Appearing Brain Areas
- WMH-DualTasker: A weakly-supervised deep learning model for automated white matter hyperintensities segmentation and visual rating prediction
- Fusing Unsupervised and Supervised Deep Learning for White Matter Lesion Segmentation
- Pathology Segmentation Using Distributional Differences to Images of Healthy Origin
- Beyond Neurofibrillary Tangles: Explainable AI for Microscopic Tauopathy Classification in Immunofluorescence Imaging
- SpurCon: Weighted Supervised Contrastive Learning for Mitigating Spurious Cues in Medical Imaging
- Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge