Keeping Only the Missingness That Travels: Radiomics–Clinical Fusion Across Hospitals

Hospital records have gaps, and the gaps carry information. A lab test is often absent because a clinician judged the patient too well to need it, or too sick to wait for it. Models that combine features computed from tumor scans with these patchy clinical tables usually handle the gaps in one of two ways. They erase the gaps, through imputation or invariance training, and so throw that information away. Or they add "was this measured?" flags and let the model use the flags freely. The second option also absorbs each hospital's documentation habits, and those habits change when the model moves to a new hospital. We propose a small fusion model for survival prediction that feeds the missingness pattern of the clinical table in as its own input. We then constrain the model's use of that pattern to the part whose relation to outcome is stable across hospitals, so that site-specific documentation quirks are discarded.

The imaging side uses radiomics: several hundred hand-designed numbers per tumor describing its shape, intensity histogram, and texture, computed from a segmented region of a CT or MRI. We use radiomics rather than deep image features because the relevant cancer cohorts contain a few hundred patients. At that size, the file's evidence favors engineered features fed to simple models. Radiomics also puts the image into a tabular form, so image and clinical data can be joined early, before any prediction is made. The question is whether they can be joined without the missing clinical entries either poisoning the combined representation or teaching the model shortcuts that break at the next hospital.

Core Claim

Informative missingness in clinical tables has two parts. The acuity part reflects how sick the patient is and should transfer across sites. The Tan et al. 4CE analysis of 69,939 COVID-19 patients across 15 sites found that non-severe patients reach maximal missingness faster than severe ones (Informative Missingness). The site part reflects local ordering policy and data-mapping errors, and should not transfer. The same study found mapping errors in 7 of 15 hospital systems.

We claim that an early-fusion survival model can separate the two. It gets an explicit missingness input, and a penalty makes the prognostic contribution of that input consistent across training sites. We expect this model to beat two alternatives on held-out sites. The first erases missingness, as in RoVTL. The second uses indicators without constraint, as a gradient-boosted tree does by default. We further claim the constrained model will be the only one of the three that stays calibrated when clinical attributes are deleted at random at test time. That deletion is not acuity-driven, so a correct model should not read it as a sign of health.

The contrast with the closest prior work is direct. RoVTL simulates random attribute dropout during contrastive pretraining so that representations do not depend on which attributes are present. It explicitly assumes missingness is roughly random and uses raw-pixel backbones rather than radiomics. The multimodal breast-recurrence model of MRRP fuses whole-slide images, ultrasound radiomics, reports and clinical data. It replaces absent modalities with learned surrogate tokens but likewise ignores the information in the pattern of absence, and it is single-center. DrFuse, TriMF and PaRaChute treat missingness as whole-modality dropout to be tolerated. PaRaChute does inject a presence indicator, but only at modality level and with no tabular data. None of these asks which parts of the missingness signal survive a change of hospital.

Research Program

The input side has two parts. The first is radiomic extraction with a standard toolkit (PyRadiomics, as in the NLST study) on the tumor region. Features are grouped into families: shape, first-order intensity, co-occurrence texture, run-length texture, and wavelet- and edge-filtered variants of each. Each family is compressed by a small learned linear projection into one token. This deliberately shrinks the imaging side to roughly the token count of the clinical side. It is our response to the dimensional-imbalance concern raised by TabMixer and the heterogeneous-fusion survey, where high-dimensional image inputs drown out a few dozen clinical variables. It avoids the heavier tensor-mixing machinery those works propose, which the sample sizes here cannot support.

The second part is the clinical table. Each clinical attribute becomes a token built from a value embedding plus an observed/unobserved embedding. Unobserved entries are left out of any reconstruction target, as in the intrinsic mask of AID-MAE. Separately, the binary missingness vector $m$ is passed to a missingness branch that emits its own log-risk term.

The model predicts log-hazard as $f(x) = h(r, v) + g(m) + u(r, v, m)$. Here $r$ is the radiomic tokens, $v$ the observed clinical values, $h$ a one- or two-layer attention encoder over all tokens, $g$ the missingness branch, and $u$ a low-capacity interaction head. The interaction head lets missingness change how radiomics is read; for example, an absent staging workup may change the meaning of tumor diameter. Training minimizes

$$ \mathcal{L} = \sum_{s} \ell_s(f) + \lambda \, \mathrm{Var}_s\big[\ell_s(f) - \ell_s(f - g - u)\big], $$

where $s$ indexes training sites and $\ell_s$ is the negative Cox partial log-likelihood on site $s$. The variance term measures how much the missingness-driven components ($g$ and $u$) improve fit, and penalizes that improvement for differing across sites. Missingness signal that helps everywhere is kept. Signal that helps only at one hospital is suppressed. $\lambda$ is chosen by leave-one-site-out validation within the training sites. The penalty needs no differentiable architecture, so we also implement it in a linear Cox model with indicator terms, alongside the small attention model. This lets us check whether any gain comes from the constraint or from the network.

A diagnostic stage runs before modeling in each cohort. We fit outcome-from-missingness-only models per site, and site-from-missingness classifiers. This shows how much prognostic and how much site information the missingness pattern holds. If missingness is not prognostic anywhere, the proposal reduces to a robustness study, and we will report that honestly.

Baselines

The naive reference points are the complete-case Cox model and the clinical-only model. The NLST radiomics study reports that a Cox model on clinical data plus shape radiomics reached a C-index of 67.06 and a mean time-dependent AUC of 71.27 on 263 complete-case adenocarcinoma patients, against 54.81 for clinical data alone. We re-run exactly that pipeline on the same complete-case subset to reproduce these numbers. We then evaluate it on the full cohort, including the patients it excluded, with mean imputation at test time as its only option. Beating it means a higher C-index on the full cohort and on held-out sites, not only on the curated subset.

The strongest practical competitor is a gradient-boosted survival tree ensemble given concatenated radiomic features, raw clinical values with native missing-value handling, and explicit indicators. Trees fit small tabular cohorts well, handle gaps natively, and exploit informative missingness for free. The file's small-cohort results point the same way: a random forest on Haralick and local-binary-pattern radiomics reached 76.5% accuracy against 51.5% for a neural network on BraTS 2019 (Brain Tumor Survival Prediction). No reported result found on Lacuna for boosted trees with missingness indicators on any radiomics survival cohort, so we establish this number ourselves. It gets the same tuning budget as our model: the same number of hyperparameter trials, selected by the same inner leave-one-site-out validation.

The strongest published missing-data fusion method is RoVTL, whose reported result is an improvement of roughly 4.5% over the prior method TIP on UK Biobank coronary artery disease classification. The file gives no absolute value. We port its random-masking pretraining and gated cross-attention onto our radiomic and clinical tokens with matched parameter count. We also include MRRP-style learnable surrogate tokens for absent CT, and imputation by chained equations (MICE) followed by concatenation.

Two ablations isolate the key ingredient. The unconstrained model sets $\lambda = 0$ and uses missingness freely. The indicator-free model drops $g$ and $u$, so missingness is only masked. The claim holds only if the constrained model beats both on held-out-site C-index and integrated Brier score, and shows the smallest risk shift under random deletion. A win restricted to within-site cross-validation would not count.

Why not just…?

Why not just use gradient-boosted trees with missingness indicators? This is the most serious threat. Within a single site we expect it to match or beat us, and that is the reason it is the main baseline. Our prediction is narrower. Trees split freely on indicators, including indicators that encode one hospital's ordering policy, so they should lose ground on held-out sites and misread random test-time deletion as health. If boosted trees match the constrained model on leave-one-site-out C-index and on the random-deletion calibration test, the architecture adds nothing. In that case we would report the simpler finding, possibly with the cross-site penalty applied to a tree ensemble through per-site reweighting.

Why not just impute with MICE and concatenate? Imputation removes the very signal we want to test. The file reports that decoupled imputation distorts covariance and yields higher reconstruction error than joint approaches (MGMC). MICE-plus-indicators is included in the experiments, so if it is enough, the comparison will show it.

Why not just discard the messy clinical table and predict from radiomics alone? The NLST result shows clinical variables add to shape radiomics. The MRRP unimodal numbers show the same pattern: clinical-only 0.596 and ultrasound-only 0.470, against 0.840 fused. A radiomics-only Cox model is included and reported, so the value of the clinical side under real gaps is measured directly rather than assumed.

Why not just use late fusion of separate radiomic and clinical models? Late averaging cannot let missingness change how an imaging feature is read, which is what the interaction head $u$ does. We include a late-fusion Cox ensemble. If it matches our model, the interaction head is unnecessary and we will drop it.

Proposed Experiments

The primary testbed is the renal cancer cohort formed by combining CPTAC-CCRCC and TCGA-KIRC. These are two consortia, and within them contributing institutions give natural site environments. CT radiomics come from segmented tumors, clinical tables from the consortium records with their natural gaps, and CT itself is absent for part of the cohort. CPTAC-PDA and CPTAC-UCEC serve as secondary cohorts. PaRaChute's full-modality C-index of 0.8367 on UCEC, and its average of 0.7488 across cohorts and missingness conditions, are reference points only. They use pathology slides and no clinical table, so they are not directly comparable. The file does not report which clinical attributes these cohorts contain or how often each is missing; confirming this is the first practical step. NLST gives a single-trial test: we rebuild the adenocarcinoma cohort without the complete-case filter, and use screening-center splits if center identifiers are available. BraTS 2019, with complete sequences, serves as a controlled sandbox where we inject structured missingness in age and synthetic clinical fields with known acuity-dependent and site-dependent components. Because the ground truth is known there, it tests whether the penalty keeps the right part.

The first decisive check is leave-one-site-out C-index, time-dependent AUC and integrated Brier score against all baselines. The second is a random-deletion stress test. At test time we delete observed clinical attributes completely at random and measure the mean shift in predicted log-risk and the change in calibration slope. An acuity-aware model should barely move, while an unconstrained indicator model should shift predicted risk toward "healthy." The third, run in the BraTS sandbox, measures whether $g$ recovers the injected acuity component and ignores the injected site component. It also sweeps $\lambda$ to show the trade-off between within-site fit and cross-site transfer. Together these checks show whether hospitals' missing data can be used as a prognostic signal without inheriting each hospital's documentation habits.