Does High Latent Dimensionality Predict Cross-Modal Transfer in Biomarker Trajectory Models?

In visual neuroscience, the brain-like models that generalize best spread their internal representations across many directions rather than compressing them into a few. Clinical prognosis has gone the other way. A patient's serial lab values or scans are usually reduced to a slope, a trajectory class, or a single baseline value. This proposal asks whether the visual finding carries over to medicine. Suppose we train many neural encoders on patients' irregular, multi-visit biomarker histories. Does the "spread" of an encoder's representation, measured without any labels, predict how well it transfers to a biomarker modality it never saw? If it does, a clinic can choose a trajectory model for a new assay or a new cohort before it has outcome labels. If it does not, and low-dimensional summaries like slopes transfer better, then we have direct evidence that a celebrated principle of neural coding is specific to vision.

The question matters because the main bottleneck in longitudinal biomarker work is not model capacity. It is that each cohort and each assay is small and siloed. The cohorts in this literature range from about 200 to a few thousand patients, and almost every published model is fit and judged inside one modality and one disease. A label-free signal that predicts transfer would let information move across modalities. A clean negative result would stop the field from importing vision-era design rules that do not apply.

Core Claim

We define effective dimensionality (ED), following Elmoznino and Bonner, as $\text{ED} = (\sum_i \lambda_i)^2 / \sum_i \lambda_i^2$. Here $\lambda_i$ are the eigenvalues of the covariance of an encoder's patient-level embeddings, so ED counts roughly how many directions carry substantial variance. Our hypothesis concerns a population of trajectory encoders pretrained on serial biomarkers from some modalities. Across that population, ED should predict linear-probe performance on held-out modalities and endpoints, and it should do so after controlling for how well each encoder performs on its own source modality. The second clause is what makes the claim non-trivial. ED might simply track overall encoder quality, in which case it adds nothing to ordinary validation. We will report the partial correlation, not the raw one.

The competing hypothesis is a serious one. Clinical trajectories are sparse, irregular, and dominated by a few physiological factors. For data like that, a low-dimensional representation may be exactly what generalizes. The shapes this literature reports are a level, a direction, an S-shaped dose-response, and a "legacy" offset that persists after normalization.

What Exists and What It Leaves Open

Three ADNI studies motivate the gap. ADNI is the Alzheimer's Disease Neuroimaging Initiative, a multi-center longitudinal cohort with serial MRI, PET, fluid biomarkers, and cognitive scores.

  • Lu et al. compress each patient's irregular multimodal history into a fixed-length vector. They use linear global and patient-specific projections, softly aligned and group-sparse. Ridge-regression MMSE error drops from 0.4188 to 0.2928 RMSE. The method is linear and is tested only within one cohort and modality set.
  • The parametric Bayesian multi-task model treats each patient's regional atrophy trajectory as a task. It shrinks noisy individual fits toward subjects who are similar on predefined biomarker kernels, which roughly halves mean absolute error relative to independent per-subject fits. It models one region at a time.
  • Chen et al. show that a random survival forest on baseline plasma p-Tau217 reaches a 4-year conversion AUROC of 0.787, against 0.758 for amyloid PET. All of their predictors are single snapshots, and they name serial trajectories as the open frontier.

None of the three asks whether a trajectory representation learned on one modality is useful on another. That transfer question is what we study.

Two vision papers supply the method.

  • The ED metric and hypothesis. Elmoznino and Bonner found that ED across 568 network layers correlates with neural predictivity: r = 0.84 for macaque IT cortex and r = 0.79 for human LO. High-ED models also reach a novel-category retrieval score (mean reciprocal rank) near 0.9, against 0.2 for low-ED models.
  • A natural pretraining objective for serial data. Halvagal and Zenke introduce latent predictive learning (LPL). It pulls representations of temporally adjacent inputs together and pushes variance apart to prevent collapse. For a patient, adjacent visits are the natural "adjacent inputs."

The longitudinal clinical papers constrain the design.

  • Slowness may be the wrong prior. The S-shaped albumin–mortality relationship in Yang et al. and the persistent hazard of decreasing hsCRP trajectories in Pan et al. show that clinically important signal lives in abrupt changes and in history. A pure slowness objective would smooth exactly that away.
  • Variance can be noise. Episodic noise from D-dimer volatility (Zhuang et al.), pulsatile insulin (Mehlig) and white-coat blood pressure (Wang et al.) means high variance is not automatically useful signal. This is the main reason ED might fail in this domain.

Research Program

Data. The primary testbed is ADNI, because it is the one resource in this literature with several serially measured modalities on the same patients. Those modalities are FreeSurfer regional volumes, voxel-based morphometry, cognitive batteries such as MMSE and ADAS, CSF and plasma markers, and PET.

The first deliverable is an audit. It will count how many participants have at least two visits per modality, and in particular how many have serial plasma draws. If serial plasma data are too thin, the held-out modality becomes CSF or PET rather than plasma. We will report the counts rather than assume them.

A secondary replication uses the critical-illness cohort studied by Salciccioli et al.. There, families of laboratory tests play the role of modalities, and the much larger patient count gives the ED–transfer correlation more statistical power.

Encoder population. We will build on the order of 100 encoders by crossing four factors:

  • Architecture: a time-aware recurrent network, a Transformer with continuous-time positional encoding, and a set encoder over (time, value) pairs.
  • Width.
  • Pretraining objective: masked-visit reconstruction, next-visit forecasting, contrastive augmentation, and an LPL-style objective that pulls a patient's adjacent visits together while a variance term prevents collapse.
  • Random seed.

Every encoder is pretrained only on the source modalities. For each one we compute ED on a fixed number of held-out patients. The number is fixed because sample covariance spectra are biased when patient counts are small, so ED estimates must be comparable across encoders; bootstrap intervals are reported. We also record the eigenspectrum decay exponent, since the visual result singled out decay near $1/i$.

Transfer evaluation. Frozen embeddings computed from the held-out modality's trajectories feed a linear or Cox probe for two targets: 4-year MCI-to-AD conversion and future MMSE.

Causal intervention. For a fixed architecture, we add a spectral penalty that pushes the embedding eigenspectrum toward a chosen decay, from strongly low-rank to near-flat. Correlation across a heterogeneous model zoo can be confounded by architecture. Moving ED within one architecture and watching transfer respond is the stronger test.

Baselines

Snapshot reference. The naive reference is no trajectory at all: a random survival forest on baseline values of the held-out modality, following Chen et al.. Their reported 4-year conversion AUROC is 0.787 for p-Tau217 and 0.758 for amyloid PET. We will rerun it on our exact patient split, because our trajectory-eligible subset will differ from their 201-participant cohort. Any trajectory method that fails to beat the rerun snapshot on the same patients is not worth its complexity.

Two-stage classical baseline. This is the strongest competitor on the clinical question:

  1. Fit a per-modality linear mixed-effects model, meaning a population trajectory plus random intercept and slope per patient.
  2. Add latent-class trajectory membership as used by Pan et al., and spline terms for curvature as in Yang et al..
  3. Feed the per-patient intercepts, slopes, curvature, and class into the same survival forest.

These features are modality-agnostic by construction, so they "transfer" for free. That makes them the honest bar for any learned encoder.

Strongest existing multimodal longitudinal methods. These are the linear enrichment of Lu et al. and the kernel-coupled Bayesian model of the multi-task paper. Lu et al. report a ridge MMSE RMSE of 0.2928. For the Bayesian model we found only the qualitative "roughly halved MAE," with no absolute figure on Lacuna. We will reimplement both on identical splits.

Fairness rules. Every learned method gets the same patients, the same per-method hyperparameter search budget, and the same probe. All model selection uses source-modality validation only. The target modality is never used.

Ablations isolating the key ingredient. First, the spectral-penalty sweep within a single architecture, described above. Second, a model-selection comparison: pick the encoder by highest ED, by best source validation score, or at random, then compare their target performance.

Success criteria. The proposal "wins" on the scientific question if two conditions hold:

  • ED shows a positive partial rank correlation with target-modality AUROC after controlling for source performance, with the confidence interval excluding zero, and the correlation replicates in the ICU cohort.
  • ED-based selection beats source-validation selection on target AUROC by more than the across-seed standard deviation.

It "wins" practically only if the selected encoder also beats the two-stage classical baseline on the held-out modality.

Why not just…?

Why not just use mixed-effects slopes and trajectory classes? This is the real threat. With a few hundred patients and three to six visits each, a slope may carry nearly all the transferable signal. The two-stage baseline is designed to settle this directly. If it matches the best encoder on held-out modalities, we will report that low-dimensional summaries suffice in this regime. That would itself answer the cross-modal question negatively. The spectral sweep sharpens this: if transfer peaks at very low ED, the clinical domain prefers compression, contrary to the visual finding.

Why not just choose the encoder with the best source-modality validation score? If ED and source performance are strongly correlated, ED adds nothing, and the partial correlation will show it. Source validation is known to be an imperfect guide to out-of-distribution behavior. The visual evidence is that ED tracked generalization to unseen categories, which in-distribution accuracy does not measure directly. Whether that holds for clinical modalities is exactly what the selection ablation tests.

Why not just extend the linear enrichment or Bayesian multi-task models? Both are strong within one modality and are included as baselines. Neither produces a representation meant to be reused on a modality absent during fitting. Lu et al.'s projections are tied to their input features, and the Bayesian model's coupling depends on predefined kernels. If either nonetheless transfers well when refit on pooled modalities, that would favor structured linear models over learned encoders, and we would report it.

Why not just pretrain with a slowness objective, since serial visits are a natural fit for LPL? The clinical record argues against assuming this. Mortality risk jumps at sharp albumin shifts (Yang et al.), and decreasing hsCRP retains elevated hazard (Pan et al.). A slowness prior could erase both kinds of signal. We include LPL-style pretraining as one family and break out results on patients whose trajectories contain abrupt changes. That tests whether slowness, the second visual principle, transfers.

Decisive Experiments

Primary test. The leave-one-modality-out ADNI study. Encoders are pretrained on MRI and cognitive trajectories and probed on CSF or plasma trajectories for 4-year conversion. Then the roles are reversed. We report AUROC and time-dependent concordance for every encoder alongside ED, source performance, and their partial correlation, plus all baselines on identical splits.

Causal test. The within-architecture spectral sweep. Transfer should rise with ED if the hypothesis holds, and fall or stay flat if not.

Replication. The ICU replication, holding out laboratory families. This checks whether any ED–transfer relationship survives a different disease, time scale, and noise profile.

Noise stress test. For a subset of patients we add synthetic transient spikes, modeled on the volatility documented for D-dimer and blood pressure. If high ED is merely absorbing noise, its transfer advantage should shrink under these spikes. If it reflects richer trajectory structure, the advantage should hold.

Decision rule. A positive result in the primary and causal tests, replicated in the ICU cohort, would give longitudinal medicine a label-free way to choose models for new assays. A null or reversed result would mark the visual dimensionality principle as modality-specific.