Prognosis from Change: Within-Patient Deformation as a Scanner-Robust Signal for Longitudinal Survival

Models that predict when a patient will reach a clinical event from a series of scans (for example, how soon a person with mild cognitive impairment will convert to Alzheimer's disease) work well inside the dataset they were trained on. Nobody has shown that they survive a move to another hospital. These are time-to-event, or survival, models: they predict how long until an event happens, while handling patients whose follow-up ends before the event is seen. The best published longitudinal model reads the raw appearance of each scan, and raw appearance is exactly what changes when the scanner vendor, field strength, or protocol changes. We propose to predict risk from how a patient's anatomy changes between their own visits, not from what any single scan looks like. Concretely, we would register each scan to the same patient's previous scan, which means computing the smooth spatial warp that aligns one to the other. We then feed the per-voxel map of local tissue expansion or shrinkage from that warp, divided by the time elapsed between visits, into a survival model.

The intuition is the logic of a "fixed effects" estimator. Most scanner artifacts are roughly constant within a patient's own sequence, so taking differences between visits cancels much of them, while the disease signal, which is progression, survives. We expect this to shrink the drop in accuracy from internal to external sites more than better architectures or augmentation do. We also expect the cancellation to fail in a predictable case: a patient who switches scanners mid-follow-up. That failure gives us a way to test the mechanism directly rather than only reporting accuracy.

Core Claim

Our central hypothesis has three parts. Survival models built on within-patient, time-normalized deformation maps will lose substantially less discrimination when moved from a training cohort to unseen cohorts and vendors than models built on absolute scan appearance. Their internal accuracy will be comparable. The gain will disappear specifically for sequences that span a scanner change.

A second, subordinate claim concerns the output layer that turns features into risk, which we call the hazard head. We will compare three heads. The first is the linear Cox model, which assumes log-risk is a weighted sum of features and that relative risks stay constant over time. The second is an unconstrained neural network. The third, an additive head with one learned shape function per feature, sits between them. It can represent the non-monotonic "too little or too much" risk curves documented for biomarkers by Anaya et al. and the plateauing tumor-size thresholds found by Shen et al.. It cannot form the arbitrary feature interactions through which site-specific shortcuts can enter.

Where This Sits

Three lines of prior work motivate the proposal. SurLonFormer links longitudinal 3D brain MRI to conversion time, using a vision transformer per visit, a causal sequence transformer across visits, and a Cox head. It reaches AUC 0.827 and C-index 0.823 on ADNI, but it was evaluated only by internal cross-validation and reads absolute appearance. Shu et al. run a convolutional recurrent network over chest X-ray series. They found it partly relied on visible support hardware, a practice-dependent shortcut that should not transfer between hospitals; this is direct evidence that appearance-based longitudinal survival models absorb non-portable cues. AMOS documents a drop of more than 20% in segmentation Dice on MRI from unseen scanners, so the risk of scanner shift is not hypothetical.

One line of work supplies the method. Temporal Flow Matching shows that modeling the difference between consecutive scans, as a velocity field, is what lets longitudinal models beat the copy-the-last-scan baseline (NRMSE 0.069 versus 0.085, while the video model SimVP scored 0.711). It uses this only to generate images and never connects it to outcomes or to site shift. We reuse the difference idea as the input to a survival model, with classical registration as the default and TFM-style learned velocities as a variant.

Other work constrains the design. Disease-driven domain generalization improves cross-cohort Alzheimer's classification across ADNI, AIBL, FHS and NACC by steering attention toward anatomical priors derived from SHAP, a method for attributing predictions to input features. It works on static snapshots and diagnostic labels; we target trajectories and censored outcomes, and we use its cohorts. Multi-Source Survival Domain Adaptation and federated causal survival analysis handle survival under shift for tabular data, using target data or cross-site statistics. Ours is single-source generalization with no target labels, which the research file identifies as an open problem. DeepJoint and InputAdapter both show that differences in observation cadence between sites, meaning how often and when measurements are taken, are themselves a source of shift. This is why we divide deformation by the elapsed interval and stress-test visit spacing explicitly.

Research Program

For each patient we take consecutive T1-weighted MRI pairs. We apply bias-field correction and affine alignment, then run nonlinear registration of each later scan to the earlier one. From the resulting warp we compute the log-Jacobian map, the per-voxel log of local volume change, and divide it by the inter-visit gap in years. This yields a sequence of "atrophy-rate maps" with explicit timestamps.

A 3D convolutional encoder embeds each map. We chose a convolutional encoder over a transformer because AMOS and Sun et al. found 3D-UNet-style encoders no worse than transformers under multi-site shift. A small causal sequence model then aggregates the visits, with elapsed time as an input so that irregular spacing is handled natively. The aggregated sequence is reduced to $K$ (around 8) risk factors $z_1,\dots,z_K$. The additive head models the hazard as

$$\lambda(t\mid z)=\lambda_0(t)\exp\Big(\sum_{k=1}^{K} f_k(z_k) + \sum_{k=1}^{K} g_k(z_k)\,\log t\Big),$$

where $\lambda_0(t)$ is an unspecified baseline hazard. Each $f_k$ is a small one-input network giving factor $k$'s shape of effect on log-risk. Each $g_k$ lets that effect drift with time $t$, which relaxes proportional hazards in a controlled, inspectable way. The model is trained end to end with the Cox partial likelihood using Efron's tie correction.

The program has three stages. We first establish internal parity with appearance-based models on ADNI. We then measure transfer to cohorts and vendors not used in training. Finally, we run mechanism tests that should break the method if our explanation is wrong.

Baselines

The naive reference is a linear Cox model on age, baseline cognitive scores, and baseline hippocampal volume. Every learned model must beat it externally, not just internally.

The strongest existing method is SurLonFormer. We will reimplement it on the same preprocessed volumes, folds, and 24-month landmark, giving it the same GPU budget and hyperparameter-search trials as our model. Its reported ADNI internal numbers are AUC 0.827 and C-index 0.823 (Liu et al.). As a sanity check, our reimplementation should land within a few hundredths of those. We found no reported result on Lacuna for any longitudinal imaging survival model on AIBL or NACC, so the external numbers will be new for every method. "Beating it" means two things: internal C-index within 0.02 of SurLonFormer, and an external drop at least half as large, with confidence intervals from patient-level bootstrap that do not overlap.

A second appearance-based competitor is the same SurLonFormer trained with intensity augmentation and per-scan histogram harmonization. This represents the "pool and augment" answer. As a further reference we report Deep Clustering Survival Machines, which report an ADNI C-index of 0.78±0.04.

The skeptic's best baseline is a Cox model on segmentation-derived annualized change, meaning hippocampal and ventricular volume change per year from standard tooling. It captures much of the "change cancels scanner effects" logic with no deep learning at all.

Four ablations isolate the key ingredient. The first gives our identical encoder and head raw scans instead of Jacobian maps, which isolates the input representation. The second removes the time normalization. The third swaps the additive head for a linear Cox head and for an unconstrained network of matched size. The fourth replaces registration Jacobians with TFM-style learned velocities.

Why not just…?

Why not just compute regional atrophy rates from segmentations and fit a Cox model? This is a real threat, because annualized hippocampal loss is a known strong predictor. The segmentation-change Cox baseline exists to settle it. If it matches our model externally, the honest conclusion is that within-patient differencing, not deep representation, carries the portability. We would report that as the main finding and present the dense model only as an upper bound. We expect the dense maps to add value where change is spatially diffuse or concentrated outside predefined regions, and the per-region contribution analysis would show whether that happens.

Why not just harmonize intensities and augment heavily, then use the best appearance model? AMOS reports a drop of more than 20% on unseen MRI scanners even with careful preprocessing. Sun et al. found that generic methods such as Mixup trailed disease-grounded training by 2.8% in target accuracy. Augmentation randomizes appearance but cannot remove a consistent vendor signature that correlates with cohort outcome rates. The augmented-SurLonFormer baseline tests this directly on our splits.

Why not just use the latest scan with a static model? Shu et al. report an ICU-admission concordance error of 0.241 for their longitudinal model, against 0.358 for DeepSurv and 0.371 for Cox on static features. Liu et al. report FPCA-Cox AUCs of 0.56 to 0.71, against 0.827 for the longitudinal transformer. Rate of change carries information. Static models also read absolute appearance, so we expect them to transfer worst. They are included as a reference.

Why not just adapt to the target site with labels? Methods like MSSDA and InputAdapter need target data, and InputAdapter needs target labels. The deployment we target has neither at launch. Adaptation is complementary, and the methods could be stacked later.

Proposed Experiments

We train on ADNI MCI participants with at least two MRI visits, with conversion to Alzheimer's disease as the event and follow-up ending without conversion treated as censoring. Two shift settings follow. In the controlled setting we hold out one scanner vendor inside ADNI. In the realistic setting we test on AIBL and on the NACC subset with longitudinal MRI and recorded conversion. Assembling the NACC subset is a feasibility risk, and we will report its size before any modeling. Metrics are Harrell's and Uno's C-index (Uno's corrects for differing censoring patterns across sites), time-dependent AUC at 24 months, integrated Brier score, and calibration slope, each reported as internal value, external value, and the drop between them.

Three decisive checks test the mechanism rather than the score. First, a probe trained to predict acquisition site or vendor from each model's learned risk factors should be near chance for ours and well above chance for the appearance models. Second, we will stratify external patients by whether their sequence spans a scanner change, using acquisition metadata. The method's advantage should vanish in the scanner-change stratum, and we will also add synthetic per-visit bias-field and contrast changes to show the same failure on demand. A uniform advantage across strata would mean our explanation is wrong even if accuracy is good. Third, a cadence stress test will subsample ADNI visits to mimic sparser external schedules. Time-normalized inputs should degrade less than the unnormalized ablation, connecting the cadence lesson from DeepJoint and InputAdapter to imaging. If the additive head transfers better than both the linear and unconstrained heads, its shape functions also provide a clinician-readable account of how atrophy rate maps to risk.