Psychiatry and neuroimaging are moving away from yes/no diagnoses toward continuous "dimensions": axes such as mood, fear, or internalizing severity that are learned from brain scans and symptom questionnaires. These axes are usually estimated in one snapshot of a cohort, then reused in longitudinal follow-up. The people who leave a follow-up study are rarely a random subset. The sickest may deteriorate, the recovered may lose interest, and the frustrated may quit. That dropout is therefore selection along the very dimension being measured. Selection on a continuous axis narrows the range of that axis. A narrower range weakens brain–symptom correlations, shrinks deviation slopes, and can rotate the directions that two disorders appear to share.
We propose a Dimensional Fragility Index: the smallest amount of outcome-dependent dropout, placed along a learned latent dimension, that would overturn a specific dimensional finding. Clinical trials already have a fragility index for binary outcomes. It counts how many patients would have to switch status to erase significance, and in carotid stenosis trials the number lost to follow-up exceeded it in every trial reviewed (P.124 Assessing the fragility index…). Dimensional findings have no equivalent. The findings we target include "depression and anxiety share one brain axis," "this canonical correlation is 0.7," and "deviation grows 0.2 per disease stage." Our index gives them an equivalent, anchored by how strongly dropout is visibly related to the dimension in the observed waves.
Core Claim
The claim is that the fragility of a dimensional finding under attrition is a property of the finding's geometry. It is not captured by fragility counts on dichotomized diagnoses, and it is not captured by missing-at-random corrections. Here "missing at random" means dropout that depends only on data already recorded. A finding can be robust as a mean difference yet fragile as an axis orientation, or the reverse. The index can be computed as an exact sensitivity bound, so we do not have to pretend the dropout mechanism is identified.
That last point is a deliberate retreat from the most obvious idea at this intersection. The obvious idea is to bolt a missing-not-at-random mask model, such as not-MIWAE, onto a multimodal normative autoencoder (mmNormVAE) and "correct" the trajectories. Here "missing not at random" means dropout that depends on values never recorded. That corrective model would be new, but the research file itself flags its weakness: the data distribution and the dropout mechanism cannot be uniquely identified without strong assumptions. A corrected estimate would therefore just restate those assumptions. A bound shows plainly how strong an assumption must be before a finding breaks.
Mechanism
Let $z$ be a participant's score on a learned dimension at a follow-up wave, observed only for those who stay. We model possible dropout with one tilt parameter $\gamma$. Retention probability is the missing-at-random model fitted from observed data, multiplied by a term that depends on the unseen current score:
$$ \Pr(\text{retained} \mid z, h) \;\propto\; \pi_{\text{MAR}}(h)\,\exp(\gamma z), $$
In this expression, $h$ is the participant's observed history and $\pi_{\text{MAR}}$ is the retention model fitted from that history. Setting $\gamma = 0$ recovers ordinary missing-at-random analysis. Positive $\gamma$ means high scorers are more likely to be retained, which matches the "healthy dropout" pattern seen in long-term physical recovery (Feuerriegel et al.). Negative $\gamma$ matches loss of the sickest patients, as in advanced cancer (Hillen et al.).
For a finding $\theta$, the Dimensional Fragility Index is
$$ \text{DFI}(\theta) = \min_{\gamma}\ |\gamma| \quad \text{such that the tilted, reweighted estimate of } \theta \text{ crosses its decision threshold}. $$
The finding $\theta$ can be a canonical correlation, a deviation slope, or the cosine similarity between two disorder axes. Its decision threshold is loss of significance, loss of replication, or the two axes becoming distinguishable.
We also report a count version. It is the number of lost participants who would need scores at a given latent quantile to produce the same flip. That version is directly comparable to the categorical index.
To make $|\gamma|$ interpretable, we borrow the logic of the confounding sensitivity tools that Meaningful Associations in the ABCD Study recommends. In that approach, the strength of an unmeasured factor is expressed in multiples of a measured one. Here we estimate $\gamma_{\text{obs}}$, the dropout tilt attributable to the previous wave's score on the same dimension. A finding with $\text{DFI} < \gamma_{\text{obs}}$ breaks if the unseen current score drives dropout no more strongly than the last visible score already does.
Research Program
The work has three parts.
The first part builds tilted estimators for three families of dimensional models that the research file documents:
- sparse canonical correlation between connectivity and symptom items (Linked dimensions of psychopathology), which reported four dimensions with $r = 0.68$–$0.71$;
- normative autoencoder deviation scores and their stage slopes (NormVAE, mmNormVAE), which reported a multimodal slope of 0.2 versus 0.12 unimodal and ADAS13 correlation $r = 0.542$;
- axis-sharing statistics such as the cosine similarity between depression and anxiety directions (Breaking the Norm).
For the latent-variable models, the tilt is applied through importance weights on the retained sample, with the latent model refit under each $\gamma$. We refit rather than reweight a frozen embedding, because selection can change the learned axes themselves.
The second part validates the index in a semi-synthetic setting where the truth is known. We take fully observed cross-sectional data, impose dropout along the learned dimensions at controlled strength, and compare the predicted flip point with the realized one.
The third part applies the index to cohorts with real follow-up loss and asks which published-style dimensional claims survive at the observed benchmark strength.
The papers cited play distinct roles:
- The Talkspace study (Gender Differences in Trajectories…) motivates the work. 18.34% of early discontinuers cited perceived ineffectiveness, which is dropout tied to the symptom trajectory itself. Its only safeguard was a comparison of completers against dropouts.
- The robot-gait meta-analysis (Moucheboeuf et al.) also motivates it: dropout handling explained nearly all of the between-study variance.
- not-MIWAE constrains the design. It shows that ignoring an informative mask pulls reconstructions toward the observed mean, with Banknote RMSE of 1.19 for MIWAE versus 0.57 for not-MIWAE. It also illustrates the identifiability problem that pushes us toward bounds.
- FLASH supplies a joint trajectory-and-dropout comparator. It assumes Gaussian markers, which we test directly against bounded questionnaire scores.
Baselines
Naive reference: complete-case analysis. We refit each dimensional model on participants retained at follow-up and report the finding as if nothing were missing. It is the default in practice and fixes the level of distortion a method must improve on. To count as beaten, a method's semi-synthetic error on $\theta$ should be substantially smaller at every imposed $\gamma \neq 0$.
Missing-at-random correction: maximum-likelihood mixed models and inverse-probability weighting. These are the strongest standard practice. Mixed models are used in the Talkspace analysis, and raked propensity weighting is used in ABCD (Dick et al.). They get the same observed history $h$ that our $\pi_{\text{MAR}}$ uses, and the same covariate set, so the only difference is $\gamma$. Our claim is not that we beat them at $\gamma = 0$, where they are correct. It is that their confidence intervals fail to cover the true $\theta$ once dropout depends on the current unseen score. The index predicts the $\gamma$ at which that failure starts. Beating them means our bound intervals keep nominal coverage wherever theirs do not.
Strongest existing missing-not-at-random corrector: not-MIWAE-style mask modeling on the normative autoencoder. We give it the same architecture, training compute and hyperparameter search budget as our refits. On Banknote it reported RMSE 0.57 against 1.19 for MIWAE (not-MIWAE). No reported result was found on Lacuna for this model on longitudinal neuroimaging or dimensional psychopathology. If its point estimate lands inside our $\gamma$-indexed bound across mechanisms, it is a useful complement. If it is confidently wrong when the dropout mechanism is misspecified, as we expect, then bounding is the safer report.
Categorical fragility index after dichotomization. We threshold the dimension at the clinical cutoff (for example PHQ-9 of 10 or more) and compute the standard index. This comparison tests whether our index carries information the categorical one lacks.
Ablation isolating the key ingredient. We apply the same tilt to a scalar sum score, or to an observed covariate, instead of to the learned latent direction. If fragility rankings of findings are unchanged, then selection along the learned axis adds nothing, and the geometric claim fails.
Why not just…?
Why not just fit mixed models with full-information likelihood? They are unbiased only when dropout depends on recorded data. The mechanisms in the file, such as deterioration-driven loss, healthy dropout, and ineffectiveness-driven cancellation, depend on the current unrecorded state (How Participant Attrition Undermines Statistical Validity). Our semi-synthetic experiment measures exactly how their coverage degrades as $\gamma$ grows. If real cohorts show $\gamma_{\text{obs}}$ near zero, we will say plainly that these models suffice there.
Why not just run a completers-versus-dropouts comparison? It shows that the groups differed at baseline. It does not show whether a canonical correlation or an axis angle survives. Conditioning on post-baseline composites can also add bias of its own (Composite variable bias).
Why not just impute (multiple imputation by chained equations or MIWAE) and then rerun the dimensional analysis? Under informative dropout these methods shrink missing scores toward the observed mean, as the Banknote and SVHN results in not-MIWAE show. That compresses the tails the dimensions depend on. The imputation baseline is run in every experiment.
Why not just a standard tipping-point sensitivity analysis on one outcome? This is the real threat, because our index is its multivariate relative. It differs in three ways. The tilt sits on a learned direction and the latent model is refit, so the axes can move. The flipped quantity can be a geometric claim rather than a mean. The benchmark $\gamma_{\text{obs}}$ comes from the same dimension one wave earlier. The ablation above settles whether this matters. If scalar tipping points rank findings the same way, we will report the simpler tool as sufficient.
Proposed Experiments
Semi-synthetic validation. Using the Philadelphia Neurodevelopmental Cohort discovery sample (n = 663) and its replication sample (n = 336), we recover the four sparse-canonical-correlation dimensions. We then impose dropout proportional to $\exp(\gamma z)$ on held-back "follow-up" symptom scores, for $\gamma$ on a grid and for each dimension.
We measure three things. The first is the error between the predicted and realized flip points for canonical correlation significance and for replication of loadings (published range 0.59–0.85). The second is the coverage of bound intervals against the missing-at-random baselines. The third is whether the three dimensions that replicated (mood, fear, externalizing) differ in fragility from the one that did not (psychosis).
Longitudinal normative deviations. In ADNI we retrain the multimodal normative model on UK Biobank healthy controls (N = 9,633) and compute stage slopes and ADAS13 correlations. We then compute the index for the multimodal-versus-unimodal advantage (0.2 versus 0.12). This asks whether the conclusion that multimodal is better survives realistic loss of progressing patients.
Real multi-wave attrition. In ABCD (n = 11,880) we estimate $\gamma_{\text{obs}}$ from each dimension's prior-wave score. We then report which brain–behavior associations have an index below $\gamma_{\text{obs}}$.
Decisive checks. Three results would substantially weaken the proposal:
- the categorical index ranks findings the same way ours does;
- the scalar-tilt ablation matches the latent-axis tilt;
- inverse-probability weighting keeps nominal coverage when dropout depends on the unobserved current score.
Where applicable, we will report the simpler method as adequate. The central positive result would be axis-level findings, such as shared internalizing directions, that are flagged fragile at plausible $\gamma$ while their mean-level counterparts appear robust.
References
- P.124 Assessing the fragility index of randomized controlled trials on carotid artery stenosis: systematic review
- not-MIWAE: Deep Generative Modelling with Missing not at Random Data
- mmNormVAE: Normative Modeling on Multimodal Neuroimaging Data using Variational Autoencoders
- Longitudinal Assessment of Intersegmental Abnormalities in the Lumbar Spine of Adolescent Competitive Alpine Skiers Over 48 Months
- Digital and AI-assisted multimodal supportive care, combining physical activity, nutrition, and pain management during chemotherapy for advanced pancreatic cancer patients: study protocol of the European multicenter randomized controlled trial of the RELEVIUM project
- Meaningful associations in the adolescent brain cognitive development study
- Linked dimensions of psychopathology and connectivity in functional brain networks
- NormVAE: Normative Modeling on Neuroimaging Data using Variational Autoencoders
- Breaking the Norm: Population-Scale Normative Modeling of Brain Structure in Depression and Anxiety
- Gender Differences in Trajectories of Depressive Symptoms Among Talkspace Clients: Naturalistic Observational Study
- Effects of robotic gait training after stroke: A meta-analysis
- An efficient joint model for high dimensional longitudinal and survival data via generic association features
- Composite variable bias: causal analysis of weight outcomes