Borrowing the Reasons Patients Skip Questions: Transferable Missingness Priors for Small PROM Cohorts

When a patient leaves a quality-of-life question blank, the blank usually means something. Someone too fatigued to finish a survey, or too impaired to perform the activity a question asks about, skips it for reasons tied to the very answer that is missing. Statisticians call this "missing not at random" (MNAR): whether a value is missing depends on the value itself. In a small clinical trial whose primary endpoint is a patient-reported outcome measure (PROM), meaning a validated questionnaire scoring symptoms or function from the patient's own perspective, this kind of skipping biases the endpoint. Yet the only models that learn the skipping mechanism from data need thousands of patients, and the standard trial remedy is to guess its strength by hand in a sensitivity analysis. This note proposes to learn why patients skip once, on large online PROM registries, and carry that knowledge into cohorts of 16 to 100 patients. The mechanism is indexed by the wording of each question rather than by its column position, so it can be applied to a questionnaire the small study wrote itself.

The core claim is that the mechanism linking a symptom's severity to the chance of not reporting it is largely a property of the question's content: a mobility item behaves differently from a mood item, and both behave differently from a demographic item. Because of that, a missingness model conditioned on a text embedding of each question can be pretrained on Parkinson's registries with tens of thousands of respondents and then frozen. In the small target study, only a handful of parameters are left to fit. If the claim holds, small PROM studies get less biased endpoint estimates and better imputations of skipped items, without fitting a deep MNAR model on 50 people and without choosing an arbitrary bias correction.

What exists and what is missing

Three strands of prior work each supply one piece. Language Modeling Screens Parkinson's Disease with Self-reported Questionnaires (Quest2Dx) turns survey rows into text sequences of question plus answer, encodes them with BERT, and represents skipped questions with a single learnable "missing token." It transfers across questionnaires that share no column schema (cross-survey AUROC 0.920 PPMI→Fox Insight, 0.952 Fox Insight→PPMI). This is the method we reuse for question-indexed representation, and it is also the closest competitor. Its training cohorts have more than 40,000 subjects, its task is binary screening, and its missing token is one shared vector. It never models why a given item was skipped, and it never estimates what the skipped answer was. not-MIWAE: Deep Generative Modelling with Missing not at Random Data supplies the statistical machinery. It jointly models data and the missingness mask, factorizing $p(x,s)=p(x)\,p(s\mid x)$, and recovers values in distribution tails where MAR models collapse toward the observed mean (Banknote imputation RMSE 0.57 vs. about 1.19 for MIWAE; Yahoo! R3 MSE 0.939). However, it fits the missingness network per dataset on dense numeric matrices. On a 50-patient cohort, its encoder, decoder, and mask model would overfit together. Can we generate portable representations for clinical time series data using LLMs? (Record2Vec) motivates the transfer bet. It shows that verbalizing ICU records with a frozen language model lets a model adapt to a new site with 16 samples. Its inputs are objective vitals and labs, though, and it does not address informative missingness.

The PROM-specific papers motivate the problem without solving it. The Fox Insight cohort paper documents 90.5% cross-sectional completion among Parkinson's participants but about 10.1% drop-off across assessments within a period, which is exactly the regime where informative skipping accumulates. An Oversampling-Enhanced Multi-Class Imbalanced Classification Framework for Patient Health Status Prediction Using Patient-Reported Outcomes handles skipped radiotherapy PROM items with iterative imputation under an implicit MAR assumption before oversampling, on cohorts of more than 2,000. Fine-tuning Zero-shot Large Language Models for Patient-reported Outcomes assumes complete questionnaires. Two papers constrain the design. Domain constraints improve risk prediction when outcome data is missing shows that external knowledge (there, known disease prevalence) is what rescues inference under selective missingness. It notes that such prevalence anchors rarely exist for subjective endpoints, so we need a different external anchor, and a large registry's learned mechanism plays that role here. Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains finds that generative augmentation fails to beat simple baselines in data-scarce settings. We therefore do not use the generative model to manufacture extra patients, only to correct estimates for the patients we have.

Core mechanism

Let $x_{ij}$ be patient $i$'s answer to item $j$, $s_{ij}\in\{0,1\}$ indicate whether it was observed, $q_j$ the item's text, and $e(q_j)$ its embedding from a frozen sentence encoder. The model is

$$ p(x_i, s_i \mid q) = \int p(z_i)\prod_j p_\theta\big(x_{ij}\mid z_i, e(q_j)\big)\, p_\phi\big(s_{ij}\mid x_{ij}, z_i, e(q_j)\big)\, dz_i , $$

where $z_i$ is a low-dimensional patient latent (overall disease burden), $p_\theta$ is an ordinal answer model, and $p_\phi$ is the skipping model. The skipping model lets the probability of answering depend on the answer itself (self-masking), on general burden, and on what the question asks. Training follows the importance-weighted, doubly reparameterized objective of not-MIWAE, but item identity enters only through $e(q_j)$. Pretraining runs on Fox Insight and PPMI Online jointly, pooling across their 7,651 and 1,387 distinct questions. In a small target cohort, $\phi$ is frozen, along with most of $\theta$. Only a per-cohort scalar offset on the skipping logit (allowing different overall compliance) and a small adapter on $\theta$ are fitted, and patient latents are inferred.

There is an identifiability caveat. MNAR mechanisms cannot be identified from observed data alone, as the not-MIWAE authors acknowledge. Our method replaces that impossibility with a testable assumption: the severity-to-skipping relation, conditioned on question content, is shared across cohorts. Large longitudinal registries give partial leverage on this assumption. An item skipped at one wave and answered at the next provides a noisy view of the skipped value, and we use these pairs both as auxiliary supervision for $\phi$ and as held-out checks. Whether such pairs exist in sufficient numbers in Fox Insight must be verified in the first month. If they are scarce, the auxiliary term is dropped, and the transfer assumption is tested only through the cross-registry checks below.

Research Program

The work has three stages. The first builds the question-indexed MNAR model and pretrains it on Fox Insight plus PPMI. We then inspect what the skipping model learned, for instance whether mobility and fatigue items carry stronger self-masking than demographic ones. Such a pattern would be a sanity check that the model captures clinically plausible structure rather than noise. The second stage tests transfer of the skipping mechanism itself. We pretrain on Fox Insight only, apply the frozen $\phi$ to PPMI items never seen in training, and ask whether it predicts which PPMI items go unanswered by whom better than a mechanism fit on PPMI subsamples. The reverse direction is also run. The third stage is the small-cohort evaluation. Target cohorts of $N\in\{16,32,64,100\}$ are drawn repeatedly from the held-out registry, and the downstream quantities a small study cares about are measured: mean item and domain severity, the difference in severity between two arms or subgroups (a stand-in for a PRO-endpoint treatment effect like the dryness score in the OASIS protocol), and per-patient imputation of skipped items. Because small-sample metrics are noisy, every configuration is repeated over at least 200 random subsamples, and we report the full distribution of errors, not single splits.

Ground truth comes from three sources of increasing realism. Semi-synthetic masking applies known self-masking mechanisms to complete records, which make up the majority given 90.5% completion. Here we know the truth exactly, but the mechanism is ours. The skip-then-answer longitudinal pairs give real but noisy truth. Yahoo! R3 is the one benchmark in the file whose test set is collected at random and is therefore free of MNAR. We use it only to confirm that our reimplementation of the not-MIWAE machinery behaves correctly and to measure how its error grows as users are subsampled to small numbers. It does not test PROM transfer.

Baselines

The naive reference point is complete-case analysis: drop patients or items with blanks and average what remains, plus mean imputation for item-level tasks. This is what many small PROM studies effectively report. It is expected to be biased toward healthier values under self-masking, and any method that does not clearly beat it on bias is not worth its complexity.

The strongest classical competitor is multiple imputation by chained equations followed by gradient-boosted trees with explicit missingness-indicator features, the pipeline family used in the oversampling PROM framework (Random Forest AUC 0.902 for prostate bowel pain on cohorts above 2,000). It runs on the identical subsamples, with the same total hyperparameter budget tuned by nested cross-validation inside each subsample. Missingness indicators give trees a way to exploit informative skipping for prediction, as Transfer Survival Boosting does by splitting on missingness. They do not, however, correct population-level estimates of the skipped values themselves.

The strongest existing method on this data is Quest2Dx with its learnable missing token, pretrained on the source registry and fine-tuned on each small target subsample with the same adapter size and optimizer budget as ours. Its reported in-domain and cross-survey numbers are AUROC 0.977/0.974 and 0.920/0.952, all at full cohort size. For the small-$N$ regime there is no reported result on Lacuna for Quest2Dx at $N\le100$, so we establish that number ourselves. For estimation-type targets, we pair it with a decoder head that predicts item answers, since the original model only screens.

The standard trial practice is delta-adjusted multiple imputation, which shifts imputed values by a hand-chosen amount to represent assumed MNAR bias. We run it at the delta values a trialist would plausibly choose, and also at the oracle delta on semi-synthetic data, which serves as an upper bound for that family. A not-MIWAE model trained only on the target subsample shows what happens when the MNAR model is fit from scratch at small $N$.

Three ablations isolate the key ingredient. The first uses the same model with the skipping term removed, which makes it an MAR version. The second pretrains a skipping model that ignores question text and shares one mechanism across all items. The third fits the full model on the target subsample alone, without pretraining.

Beating the field means three things. On semi-synthetic and longitudinal-pair truth, our model must reduce absolute bias in cohort mean severity relative to MICE-plus-indicators and to target-only not-MIWAE at $N\le64$, with the reduction holding across most subsamples rather than on average only. It must match or exceed oracle-free delta adjustment without being told the delta. And on screening AUROC it must not fall measurably below fine-tuned Quest2Dx. Our claim is about estimation, so equal discrimination with lower bias counts as a win.

Why not just…?

Why not just fine-tune Quest2Dx with its missing token on the small cohort? This is the real threat for prediction tasks. A pretrained missing token may already encode "skipped usually means worse," and if discrimination is the only goal, it may suffice. We concede this possibility in advance and do not claim AUROC gains. The difference shows up in estimation: a single shared token cannot express that skipping a stair-climbing item means something different from skipping an income item, and it produces no estimate of the skipped value. The question-agnostic ablation tests exactly this. If sharing one mechanism across items is as good as conditioning on question text, the central claim fails and we will say so.

Why not just use multiple imputation with missingness indicators and a tree ensemble? This pipeline is strong and cheap, and at $N\approx100$ it may well match us on classification. Its imputation step still assumes MAR, and the not-MIWAE results show MAR-based imputers collapsing toward the observed mean under self-masking. So the specific prediction is that this pipeline will look fine on AUROC while showing systematic bias in cohort mean severity on the semi-synthetic and longitudinal-pair evaluations. If it shows no such bias, then skipping in these registries is close to ignorable, and the problem we target is smaller than we think. That would be a useful negative result.

Why not just run a delta-adjusted sensitivity analysis, as trials already do? Delta adjustment is principled and transparent, but the delta is a guess, usually one value for an entire questionnaire. Our proposal can be read as estimating item-specific deltas from a large registry instead of assuming them. The oracle-delta comparison settles whether the learned mechanism approaches the best possible hand choice, and the realistic-delta comparison measures what trialists currently lose.

Why not just ask a large language model what the patient would have answered? Zero-shot imputation needs no training, but the file gives two reasons for doubt. Prompting underperformed LoRA fine-tuning on PROM depression detection (AUC 0.737–0.746 vs. 0.750, per the LLM PROM study). And the Roux-lette line of work reports that language models fill gaps in incomplete records with plausible fabrications. We include a zero-shot imputation run with Llama-class models as an extra baseline on the semi-synthetic truth, at no cost to the design.

Decisive checks

Three results decide the proposal. First, the cross-registry transfer of the skipping model: when trained on Fox Insight, $\phi$ must predict PPMI item non-response, measured by held-out log-likelihood of the mask, better than a mechanism fit on a 64-patient PPMI subsample. If it does not, skipping is not a portable property of question content, and nothing downstream should be trusted. Second, the bias curve: absolute bias in estimated cohort severity plotted against $N$ from 16 to 100, for our model, MICE-plus-indicators, target-only not-MIWAE, and the MAR ablation. The proposal succeeds if our curve stays lowest at small $N$ and the MAR ablation reproduces the classical pipeline's bias, which would show that the gain comes from the skipping model and not from pretraining alone. Third, the question-agnostic ablation must lose to the full model on item-level imputation error for the items with the strongest learned self-masking. Only that result would show that reading the question is what makes the borrowed mechanism work.