Patients who share a diagnosis, or even the same disease-causing mutation, often lose brain tissue in different places and at different speeds. Models that forecast a patient's future 3D MRI scan from earlier ones learn a single "average" way the brain changes over time. They need hundreds of patients to train, and rare genetic dementias, where clinical trials most need individual forecasts, can offer twenty. We propose to learn a small library of reusable progression modes on large Alzheimer's cohorts. Each mode is a 3D deformation that moves tissue in a characteristic pattern, for example hippocampus-led shrinkage or striatum-led shrinkage. A new patient in a tiny cohort is then described by only a handful of numbers: how much of each mode their brain follows, and how far along they are. Forecasting becomes the task of fitting those few weights from one or two scans, not training a 3D network from scratch.
The bet is that heterogeneity is not something to average away. It is the structure that makes transfer to small samples possible. Different subtypes share building blocks, and a rare genotype may recombine them in proportions never seen in the source cohort. We will test this on held-out genetic frontotemporal dementia subgroups and on deliberately withheld Alzheimer's subtypes. The comparison includes the naive baselines that defeat most longitudinal imaging models and the regional-volume statistical models that a skeptic would reach for first.
Core Claim
A continuous-time deformation model can transfer to target cohorts of $K \le 20$ subjects if its velocity field is factored into a shared population term plus a low-dimensional, patient-weighted combination of subtype modes learned on a large source cohort. A velocity field here is the rule that says, at every voxel and moment, which way tissue is moving. Concretely, we model the field that warps a patient's baseline scan toward their future anatomy as
$$ v(x, t \mid w, s) = v_0(x, \tau(t, s)) + \sum_{k=1}^{M} w_k \, u_k(x, \tau(t, s)), $$
where $x$ is a voxel location and $t$ is real elapsed time. The term $s$ is a patient-specific stage offset, and $\tau(t,s)$ is the resulting disease time. The term $v_0$ is the shared population field. The $u_k$ are $M$ (roughly 4 to 8) learned mode fields, and $w \in \mathbb{R}^M$ are nonnegative patient weights. In the target cohort, the networks producing $v_0$ and $u_k$ are frozen. Only $(w, s)$ per patient, plus a cohort-level prior over $w$, are estimated. That is at most about ten numbers per patient instead of millions of network weights. This is what should keep the model from falling below zero explained variance, which is where image-based transfer collapses at $K \le 20$.
Where This Sits
Three lines of work each cover part of this problem. Continuous-time 4D forecasters treat a scan series as an ordinary differential equation in time. CRONOS uses flow matching with Fourier time embeddings. Temporal Flow Matching does the same, and Longitudinal Flow Matching adds piecewise-quadratic paths across multiple visits. Flow matching means training a network to predict the velocity that carries one sample to another. These models finally beat the "copy the last scan" baseline. However, all of them assume one population-wide dynamic, and none addresses tiny training cohorts. TimeFlow supplies our deformation machinery: a 3D U-Net with continuous time conditioning that trains from only two scans per subject. Because it models change as a smooth warp, it fits atrophy, though not new lesions. It has no notion of subtype. Closest in spirit is Δ-LFM, which learns patient-specific latent flows and finds that disease status self-organizes in its latent space. Its ArcRank loss fixes each patient to a single direction, with severity growing monotonically along it. That rules out a patient who mixes two atrophy patterns or shifts between them. It also trains only on large cohorts.
Heterogeneity research supplies the motivation and the evaluation targets. SuStaIn separates subtype ("what pattern") from stage ("when") using regional volumes. It finds three sporadic Alzheimer's subtypes and two C9orf72 subtypes, and it separates GRN from MAPT carriers at 95% accuracy. It works on pseudo-time inferred from cross-sectional data and pre-extracted regional summaries, so it cannot forecast an actual scan at an actual date. Family studies make the same point at the genetic level. Carriers of identical EXT1/EXT2 variants or FLNA abnormalities diverge sharply, which argues for continuous mixtures over hard subtype labels.
Small-sample work supplies both the method template and a hard constraint. Meta-matching trains 3D CNNs on UK Biobank to predict many phenotypes and reuses the correlated ones on small cohorts, beating classical transfer and Elastic Net. Its absolute coefficient of determination is near zero or negative at $K \le 20$. We borrow the idea of reusing a library of source-learned predictors. We change what is reused, from static phenotype predictors to dynamic progression modes, and we shrink what is adapted to a few interpretable coefficients. Finally, the Time-Aware Multi-View MRI benchmark shows foundation models localizing change at up to 97.3% accuracy but quantifying it below 16%. A deformation-based forecaster gives volume change directly from the warp's local expansion factor (its Jacobian determinant). This lets us probe that gap as a secondary result.
Research Program
The first stage learns the mode library on pooled longitudinal Alzheimer's data from ADNI, OASIS-3 and AIBL. We start from a TimeFlow-style time-conditioned 3D U-Net. Its output is split into a shared head producing $v_0$ and $M$ mode heads producing $u_k$. Per-subject weights $w$ and stage $s$ are amortized: a small encoder infers them from the baseline scan and the first follow-up. The network is trained to register every observed later scan. Two regularizers keep the modes interpretable and reusable. One encourages spatially sparse, mutually decorrelated $u_k$. The other is a simplex-style penalty on $w$, so that subjects cannot spread weight over every mode. We will check whether the learned modes line up with the SuStaIn Alzheimer's subtypes on ADNI, but we will not supervise on them.
The second stage is adaptation to a target cohort. Everything learned in stage one is frozen. For each target subject we fit $(w, s)$ by minimizing registration error between the baseline warped forward and the subject's observed follow-up. A Gaussian prior on $w$ is fitted jointly across the $K$ subjects. The prior lets a genotype group pool information: twenty C9orf72 carriers jointly estimate where their group sits in mode space. Forecasting then integrates the field to an unseen future date. A variant of the method fits a single low-rank correction to $v_0$. This tests whether a sliver of extra target-specific capacity helps or hurts at each $K$.
The third stage is analysis. We read off each target group's position in mode space and test whether it reflects real biology. GRN and MAPT carriers should separate, and C9orf72 should show two clusters. We also project Jacobian-derived volume changes into the quantitative questions of the Time-Aware benchmark for its ADNI and OASIS-2 portions. This asks whether a 3D forecaster answers "how much did it grow or shrink" where 2D-slice language models fail.
Baselines
The naive reference points are copying the last observed scan forward and linear extrapolation of the observed deformation. The latter means taking the warp between a subject's two observed scans and scaling it by elapsed time. Both cost nothing and cannot overfit. The literature shows copying the last scan is hard to beat: on Lumiere, Temporal Flow Matching reports 0.069 normalized root-mean-square error against 0.085 for it, while SimVP scores 0.711. Linear delta extrapolation is the sharper version for slow atrophy, and the one we must beat at every $K$. Beating it means lower held-out regional volume-change error and lower change-specific error (Δ-RMAE), with a paired test across subjects.
The strongest existing image-level method is Δ-LFM. On ADNI it reports 30.59 dB PSNR against 29.72 dB for MambaControl, and a 21% relative Δ-RMAE reduction over competitors. We first reproduce it on the full ADNI split to confirm our implementation lands near 30.59 dB. Then we transfer it to each target cohort by fine-tuning with the same subjects, compute budget and hyperparameter search as our adaptation step. TimeFlow fine-tuned on the target is a second strong transfer baseline, since it shares our backbone. No reported number was found on Lacuna for TimeFlow forecasting accuracy, so we establish its numbers ourselves. Beating these models means lower forecast error at $K \in \{5, 10, 20\}$ while staying within noise of them at $K \ge 50$.
The strongest non-imaging competitor is SuStaIn fitted on FreeSurfer regional volumes, combined with a per-subtype linear mixed-effects model of regional volume over time. Mixed-effects means each subject gets their own slope and intercept, shrunk toward their subtype's average. On GENFI, the published reference point is SuStaIn's 95% GRN-versus-MAPT discrimination. We compare on the same regional volume-change targets, scoring our forecasts by integrating Jacobians over the same parcels.
Two ablations isolate the key ingredient. The first replaces the continuous mixture $w$ with a one-hot subtype code, assigned by SuStaIn or by nearest mode. The second replaces the learned modes with principal components of source-cohort deformation fields. The first tests whether continuous mixing matters. The second tests whether a learned time-dependent flow adds anything over a static linear basis.
Why not just…?
Why not just fit SuStaIn plus mixed-effects models on regional volumes? This is the real threat. Twenty subjects times a few dozen regional volumes is exactly the regime where low-dimensional statistics shine. Meta-matching's 3D CNNs beat FreeSurfer Elastic Net on static phenotypes, but that evidence does not transfer automatically to forecasting change. Our experiment scores both methods on identical parcel-level targets. If the regional model ties us there, we will report it plainly. The remaining case for the volumetric model would then rest only on what parcels cannot express: within-region patterns (Δ-RMAE on voxels), forecasts at real dates rather than pseudo-time, and the ability to produce an image. If it beats us on parcels at $K \le 20$, the central claim fails.
Why not just fine-tune an existing 4D forecaster on the small cohort? Meta-matching found classical transfer far behind reusing correlated source predictors, with a 136% gap in variance explained at 100 HCP-YA subjects. It also found every image-based method near zero explained variance at $K \le 20$. That suggests full fine-tuning of a 3D network at twenty subjects will mostly memorize scanner idiosyncrasies. The fine-tuned Δ-LFM and TimeFlow baselines, plotted as error against $K$, test this directly. The interesting quantity is the crossover $K$ above which fine-tuning catches up.
Why not just condition one flow model on the subtype label? This is cheaper, and on Alzheimer's data with SuStaIn labels it may match us. It breaks when the target cohort's patterns are not among the source subtypes, and C9orf72's subcortical pattern is a candidate case. Our withheld-subtype experiment and the one-hot ablation settle this. If one-hot conditioning matches continuous mixing even on withheld subtypes, we will adopt the simpler model.
Why not just ask a vision-language model? The benchmark already shows these models quantify change below 16% of the time. They also degrade when given more views, so we treat them only as a secondary comparison on quantification questions.
Proposed Experiments
The main experiment uses GENFI genotype subgroups (GRN, MAPT, C9orf72) as target cohorts. We subsample $K \in \{5, 10, 20, 50\}$ training subjects over many random draws and forecast each held-out subject's last visit from their first two. The mode library is trained only on ADNI, OASIS-3 and AIBL. A second, fully controlled experiment runs SuStaIn on ADNI, removes all subjects of one subtype (for instance subcortical) from source training, and treats that subtype as the rare target. This gives a case where the target pattern is unseen in the source.
Primary metrics are regional volume-change error and voxel-level Δ-RMAE. Secondary metrics are NRMSE against copy-the-last-scan, and whether the fitted $w$ recover genotype (accuracy against SuStaIn's 95% reference) and C9orf72 bimodality. A tertiary metric is accuracy on the Time-Aware benchmark's quantification questions.
Three outcomes decide the proposal. The method must beat linear delta extrapolation at $K \le 20$, or it adds nothing practical. It must at least match the regional mixed-effects model on parcel targets, or the volumetric component is not earning its cost. And continuous mixing must beat both one-hot conditioning and principal-component modes on the withheld-subtype target, or the heterogeneity mechanism is not what drives transfer. If all three hold, small rare-disease cohorts could receive individual, date-specific 3D forecasts without being large enough to train a 3D model.
References
- CRONOS: Continuous time reconstruction for 4D medical longitudinal series
- Temporal Flow Matching for Learning Spatio-Temporal Trajectories in 4D Longitudinal Medical Imaging
- Longitudinal Flow Matching for Trajectory Modeling
- Temporal Conditioning for Longitudinal Brain MRI Registration and Aging Analysis
- Learning Patient-Specific Disease Dynamics With Latent Flow Matching For Longitudinal Imaging Generation
- Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference
- EXT1 and EXT2 Variants in 22 Chinese Families With Multiple Osteochondromas: Seven New Variants and Potentiation of Preimplantation Genetic Testing and Prenatal Diagnosis
- Cases report: MRI findings of asymptomatically familial subependymal heterotopia with filamin A gene abnormality
- Translating phenotypic prediction models from big to small anatomical MRI data using meta-matching
- How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning?