Condition on Why a Scan Was Skipped, Not on Whether It Was

Hospitals do not skip MRI sequences at random. Contrast-enhanced scans are left out when a patient's kidneys cannot tolerate the dye, when a referring note says "no mass suspected," or when local protocol reserves contrast for large lesions. Brain-tumor segmentation models built to handle missing sequences learn from such data anyway. They either assume the gaps are accidental, or they quietly absorb the local habit of which scans get ordered. The second failure is subtle. If a model learns at one hospital that "no contrast scan" usually means "little enhancing tumor," it will under-segment enhancing tumor at a hospital that skips contrast for unrelated reasons. A different scanner at the new site makes this worse, but even a good fix for scanner differences does not remove it.

This note proposes a simple, testable principle for this setting: a missing-modality segmenter should be given the recorded clinical reasons that drive acquisition decisions, and should never see the missingness pattern itself. When those reasons explain the acquisition policy, the prediction from the observed scans plus the reasons stays the same across hospitals, whatever each hospital's ordering rules are. The pattern of which scans are present carries no such guarantee. We build students for each subset of sequences, distilled from a full-sequence teacher under this principle. We add a check, run on labeled source data, of whether the recorded reasons are sufficient. We then test transport across simulated and real policy shifts, crossed with real institution and scanner shift.

Core Claim

Let $X_O$ be the observed sequences for a patient, $X_M$ the missing ones, $R$ the pattern of which sequences were acquired, $C$ the recorded clinical context available at ordering time (renal function, referral indication, prior-imaging summary), and $Y$ the voxel labels. Causality matters in medical imaging argues that segmentation is a causal task. The image drives the annotation, so the full-image labeling rule $P(Y \mid X_O, X_M)$ is a reasonable candidate for an invariant across sites. Suppose the acquisition decision depends only on what the clinician could see, that is, $R \perp (X_M, Y) \mid X_O, C$. This is the kind of graph condition under which Graphical Models for Inference with Missing Data shows distributions are recoverable from incomplete data. Under that assumption,

$$ P(Y \mid X_O, C, R) \;=\; \int P(Y \mid X_O, X_M)\, p(X_M \mid X_O, C)\, dX_M , $$

and the policy $R$ drops out of the right-hand side. If a site's policy changes, only the distribution of $(X_O, C)$ changes. That is an ordinary covariate shift, which importance weighting handles. The labeling rule itself does not move. In contrast, $P(Y \mid X_O, R)$ without $C$ depends on the source site's policy through $p(X_M \mid X_O, R)$. Mask-conditioned models therefore transport badly exactly when policies differ.

This reconciles a direct contradiction in the file. Causal Representation Learning from Multimodal EHRs under Non-Random Modality Missingness gates representations on the missingness pattern and reports up to a 13.8% improvement on MIMIC-IV outcomes. MultiModN bypasses missing modules precisely because mask-conditioned fusion fails when train and test missingness patterns are flipped. Our reading is that the mask helps in-domain because it stands in for $C$, and hurts under shift because it also encodes the local policy. Using $C$ directly should keep the in-domain benefit while removing the policy dependence.

Research Program

The method has three pieces.

First, a full-sequence teacher is trained on source patients who have all four BraTS sequences. Second, for each subset $S$ of sequences we train a student on $(X_S, C)$. We construct its training cases from full-sequence patients by deleting the complementary sequences, so the student never receives $R$ as input. The student distills from the teacher's soft predictions, reusing the prototype-relation loss of ProtoKD and the Hölder divergence of Robust Divergence Learning as the transfer mechanism; these papers supply the distillation machinery. Both of those methods sample which sequences to drop uniformly at random. Instead, we weight each synthetic subset case by an estimated target-site propensity $\pi_t(S \mid X_S, C)$. Where target labels are absent but target acquisition logs exist, this is fit on unlabeled target metadata. This weighting is the covariate-shift correction implied by the equation above.

Third, and cheapest, a policy-sufficiency check uses source labels. The assumption $R \perp Y \mid X_O, C$ has a testable consequence: on labeled source data, a classifier predicting $R$ from $(X_O, C)$ should gain nothing when also given summaries of $Y$, such as enhancing-tumor volume. A significant gain flags that the recorded context misses part of the policy. In that case we report bounds from a sensitivity parameter on the residual dependence rather than claiming invariance.

The full condition also requires $R \perp X_M \mid X_O, C$, and this part is untestable with incomplete data, consistent with the non-identifiability results summarized in the file. So it is tested only in the semi-synthetic regime, where we control it.

Baselines

The naive reference is a 3D U-Net trained with uniform random modality dropout, ignoring both $R$ and $C$. This is the standard recipe, and equivalent in spirit to MultiModN's bypass. It is correct only if acquisition is random.

The strongest existing missing-modality segmenter in the file is Robust Divergence Learning. It reports an average whole-tumor Dice of 88.2% on BraTS 2018, and a 6.2% tumor-core improvement over the next competitor in the T1-only case. ProtoKD reports unimodal Dice gains of 4.3%, 0.9%, 3.7% and 2.1% for T1, T2, T1ce and FLAIR. No reported result was found on Lacuna for any missing-modality method on BraTS 2021, or under cross-institution policy shift, so we rerun both methods ourselves.

The policy-aware competitor is a mask-conditioned model in the style of CRL-MMNAR. It uses pattern-embedding gating, trained on the naturally occurring source patterns, with the cross-fitted rectifier where patterns have common support.

All methods share the same backbone, the same 3D U-Net teacher (the strong CNN default on AMOS), identical data splits, and the same number of hyperparameter trials per method on a source-only validation set. An oracle trained on target-site patients with target policy bounds the achievable gap.

Ablations isolate the key ingredient. We test $C$ without propensity weighting, weighting without $C$, and $C$ plus the mask. The last of these tests whether reintroducing $R$ hurts transport even when $C$ is present.

Beating the baselines means the following. On target sites with a shifted policy, enhancing-tumor Dice and signed enhancing-tumor volume bias in contrast-missing patterns must beat both the dropout and mask-conditioned models, by more than three times the seed standard deviation. The method must also close at least half the gap to the oracle, while losing no more than one Dice point in-domain relative to mask conditioning.

Why not just…?

Why not just train with random modality dropout and trust the observed pixels? This is the real threat. If the remaining sequences already determine the labels well, the policy barely matters and our method is unnecessary. The experiment settles it directly. We sweep how strongly the simulated policy depends on tumor-related context. If the dropout baseline's target-site error stays flat across the sweep, we report that the effect is negligible for segmentation and stop there. We expect the effect to concentrate in enhancing tumor, which is nearly invisible without contrast.

Why not just append the missingness mask? It does help in-domain, as Adapting Fairness Interventions to Missing Values shows that missingness indicators beat impute-then-classify. But it carries the source policy, and MultiModN documents collapse when patterns flip. Our mask-conditioned baseline and the "$C$ plus mask" ablation measure that cost directly.

Why not synthesize the missing sequence and run the full teacher? Under our assumption, a synthesizer conditioned on $C$ targets the same quantity, so this is a legitimate alternative and we include a $C$-conditioned synthesis baseline. A single synthesized volume replaces the average in our equation with one sample. Unbiased Missing-modality Multimodal Learning also documents asymmetric generation bias across modalities. If synthesis matches our students, we will report that the gains come from conditioning on $C$, whatever the delivery mechanism.

Why not fine-tune on each target hospital? That requires labeled target scans under the new policy, and it changes certified weights. The oracle arm shows what fine-tuning buys. Our method needs only target acquisition metadata.

Proposed Experiments

The primary testbed is BraTS 2021 (benchmark, 2,040 patients). We hold out groups of contributing institutions where site identifiers can be recovered. Otherwise we cluster by acquisition characteristics, which provides real scanner shift. On top of this we impose simulated acquisition policies, because BraTS records none.

The context $C$ is a noisy pre-scan suspicion score derived from tumor extent plus a renal-risk covariate, with a controlled noise level. Source policies omit T1ce mainly when suspicion is low. Target policies either omit it for renal risk alone or omit it universally. A strength parameter $\gamma$ adds direct dependence of omission on true enhancing volume beyond $C$, deliberately violating the assumption. This lets us check that the sufficiency test fires when $\gamma>0$, and that the invariance gains degrade as $\gamma$ grows.

To show that policy correction and scanner correction address different problems, we cross policy shift with institution shift. We also apply OptTTA test-time augmentation to every method. Calibration within each missingness pattern is reported alongside Split Platt Scaling from Avoiding Catastrophic Referral Failures. Confident under-segmentation is the clinically dangerous failure here.

A secondary testbed checks whether the principle survives real, unsimulated ordering policies. We use eICU with leave-hospitals-out splits, the virtual-modality grouping of CRL-MMNAR, and admission diagnosis and acuity scores as $C$. Domain Adaptation under Missingness Shift serves as a baseline there. It assumed random underreporting and itself found nuanced eICU results consistent with informative missingness, but reported no number for them.

The decisive checks are as follows. In the simulated BraTS regime with $\gamma=0$, the context-conditioned student should match the oracle on enhancing-tumor Dice under the flipped policy, where the mask-conditioned model does not. The sufficiency test's false-alarm rate should stay near nominal at $\gamma=0$ and its power should rise with $\gamma$. On eICU, ranking methods by worst-hospital AUROC should reproduce the BraTS ordering. If it does not, the recorded context in real records is insufficient, and that is itself a reportable finding.