Clinical prediction models are usually built on records with gaps, such as labs never drawn or vitals never charted. Standard practice fills those gaps with multiple imputation: the missing values are guessed several times from the observed ones, so that uncertainty about them is kept. The textbook advice is to include the outcome in the imputation model. On the training set that advice is sound, and it is exactly what keeps a rare sick patient's imputed lactate looking like a sick patient's lactate rather than the healthy majority's. At deployment, though, the outcome is unknown. A new patient's gaps get filled by a model that has no label to condition on, and in a cohort that is 95% negative, its default guess is "negative-looking." The classifier therefore learns from training positives whose imputed values carry the disease signature, and is then scored on test positives whose imputed values were pulled toward the majority. We argue this train–test asymmetry is a specific, measurable mechanism behind the precision–recall collapse seen on rare clinical events. It should be worst exactly where clinical data are hardest: high missingness, rare outcomes, and missingness that depends on how sick the patient is.
We propose to measure this asymmetry directly and to remove it with class-hypothesis multiple imputation. Each test patient is imputed separately under the hypothesis "this patient has the outcome" and under "this patient does not." The risks from those two sets of imputations are then combined using weights estimated from what was observed, including which tests were ordered. Put in two sentences: label-informed imputation helps rare-class training but silently disappears at test time. Imputing under both label hypotheses and pooling by an estimated posterior restores the match, and the gap between the two hypotheses' risks tells a clinician when a missing lab is actually deciding the call.
Core Claim
The work closest to this question either assumes labels are available when imputing or ignores the test-time step. Imputation of missing data with class imbalance using conditional generative adversarial networks (CGAIN) conditions an adversarial imputer on the label to stop majority-class "wash-out" of minority records. It reports lower minority-class reconstruction error, but it requires $Y$ at imputation time, was tested only under completely random missingness, and evaluates reconstruction error rather than deployed classification. Oversampling and imputation for imbalanced missing data (OsImp) corrupts copies of minority rows with artificial gaps and imputes them alongside the real data in one deterministic pass. It neither pools across multiple imputations nor specifies how test patients are imputed. MICE and ADASYN for Missing Data Imputation and Imbalanced Data Handling on Heart Disease Classification imputes with chained equations and then oversamples, with no indication that imputation was fit inside cross-validation folds. These papers motivate the problem. None of them separates how training rows are imputed from how deployment rows are imputed, and that separation is our claim.
Concretely, let $x_o$ be a test patient's observed values and $r$ its missingness pattern (which entries are absent). Let $\tilde{x}^{(m,y)}$ be the $m$-th of $M$ imputations drawn from a chained-equations model run with the label fixed to $y \in \{0,1\}$, and let $f$ be the downstream classifier trained on label-informed training imputations. The pooled risk is
$$\hat{p}(x_o, r) = \sum_{y \in \{0,1\}} w_y(x_o, r)\,\frac{1}{M}\sum_{m=1}^{M} f\big(\tilde{x}^{(m,y)}\big),$$
where $w_y(x_o,r)$ is a calibrated estimate of $P(Y=y \mid x_o, r)$ from a model that sees only observed values and missingness indicators. Two properties matter. First, the rare hypothesis always gets $M$ draws instead of the handful it receives when the label is itself imputed at its low posterior probability. Second, the weights can absorb informative missingness, since in intensive care a lab is often ordered because the clinician suspects the rare outcome. The spread $\Delta = |\bar f^{(1)} - \bar f^{(0)}|$, where $\bar f^{(y)}$ is the mean risk over the $M$ imputations drawn under hypothesis $y$, gives a per-patient measure of how much the decision depends on unobserved data.
Research Program
The program has three parts. The first is diagnostic. On every cohort we fit the imputer inside each training fold. We then measure the distributional gap between imputed features of training positives (imputed with $Y$) and test positives (imputed without $Y$), using a two-sample classifier test on the imputed cells only, and relate that gap to the loss in area under the precision–recall curve (AUPRC). If the gap is small or does not track performance loss, the premise is wrong and we stop there.
The second part builds class-hypothesis imputation on standard chained equations (MICE, multivariate imputation by chained equations, as described in Clinical predictive modelling of post-surgical recovery in individuals with cervical radiculopathy: a machine learning approach) and on its random-forest variant MissForest. We choose these rather than deep generators because Comparison of Imputation Strategies for Incomplete Electronic Health Data found them more stable than GAIN and MIWAE under realistic missingness on clinical data. That result constrains our design toward iterative imputers.
The third part treats pooling for threshold-based decisions. Rubin's rules, as used in MatchThem, pool approximately normal estimates and say nothing about where to set an alert threshold. We will therefore choose thresholds on held-out folds using the full pooled procedure. We will report how often a patient's alert decision flips across imputations, and test whether a high $\Delta$ identifies patients for whom a recorded value, once revealed, changes the decision.
This last test also addresses the warning in Prediction-based Inference in Electronic Health Record (EHR)-linked Biobanks. That paper showed that plugging in single machine-learning predictions produced a 100% Type I error rate, and that prediction-based corrections fail when prediction errors depend on missingness. We take this as a constraint: our weights must be calibrated conditional on the missingness pattern, and we check this directly rather than assume it.
Baselines
The naive reference point is median imputation followed by a class-weighted gradient-boosted classifier. This is the cheapest pipeline a practitioner would ship. Correlation visualization under missing values shows that mean imputation distorts covariance structure, but there is no reported AUPRC for this pipeline on MIMIC-III on Lacuna. We will run it with the same folds and tuning budget as everything else. Its role is to show how much any imputation is worth here.
The strongest competitor on deployment-faithful prediction is XGBoost with native missing-value routing, explicit missingness indicators, and class weights, tuned by the same nested search as every other arm. It is strong because missingness indicators let it exploit informative missingness directly, without any imputation model. The closest reported number is from Machine learning vs. traditional regression analysis for fluid overload prediction in the ICU: XGBoost reached AUROC 0.78 with positive predictive value 0.27 at 12.5% prevalence. That cohort is not public, so the figure is context rather than a target. On MIMIC-III, MIMIC-Extract reports mortality AUROC between 0.85 and 0.90, with intervention AUPRC "often below 0.50." We reproduce its tasks and cohort definitions. To count as beating this baseline, class-hypothesis imputation must improve AUPRC with a paired-bootstrap confidence interval excluding zero on at least the 5% and 1% prevalence settings, without worse calibration.
The strongest existing imputation method for this question is standard label-including MICE with the label itself imputed at test time, using the same $M$, the same predictors, and the same downstream classifier. This is what a careful statistician would already do, and it targets the same marginal distribution as our method. The two literature-specific baselines are the sequential MICE-then-ADASYN pipeline, reproduced with fold-internal fitting to check whether the reported 97.0% Random Forest accuracy on UCI Heart Disease survives, and OsImp, whose reported Ionosphere sensitivity is 0.714 at 90% missingness. CGAIN is included only on the training side, because it cannot impute unlabeled test rows. Its reported minority reconstruction error is 0.2462 on Default Credit Card at 10% minority, but reconstruction error is not our metric.
Three ablations isolate the key ingredients. The first gives each hypothesis draws in proportion to its estimated posterior rather than $M$ each. The second removes missingness indicators from the weight model. The third imputes training rows without $Y$, which is symmetric but majority-washed and so removes the asymmetry by giving up the minority signature.
Why not just…?
Why not just impute the label along with the features at test time, as standard MICE already allows? This is the most serious threat. With enough imputations, imputing $Y$ jointly targets the same marginal distribution as our mixture. Our advantage should come from variance: at a 2% posterior, five joint draws almost never visit the positive hypothesis. We will therefore sweep $M$ from 5 to 100 at matched compute. If joint imputation matches us by $M=20$, the honest conclusion is that the fix is "use larger $M$." In that case the contribution narrows to the diagnosis and the $\Delta$ signal, and we will say so.
Why not just use XGBoost with missingness indicators and skip imputation? It sidesteps the asymmetry entirely, because nothing is imputed. It may well match us on discrimination at moderate missingness. The Sikora results show it still yields low positive predictive value on a rare ICU outcome. Our experiments test whether imputing under both hypotheses adds signal when key labs are missing for most positives. If it does not, imputation-based pipelines have no case on these tasks.
Why not just weight minority rows more heavily inside each MICE regression? This is the "cost-weighted MICE" idea the literature flags as untried. Once $Y$ is a predictor in each conditional model, weighting mainly changes the slopes shared between classes rather than the class shift. More importantly, weighting does nothing about the test-time absence of $Y$. We include it as a baseline. Our prediction is that it helps training-side reconstruction error but leaves the train–test gap and AUPRC largely unchanged.
Why not just drop incomplete records? In both the biobank and ICU studies above, complete-case analysis discarded scarce positives and kept a systematically different, more heavily monitored group. We report it once at each prevalence to quantify that loss.
Proposed Experiments
We use MIMIC-III (MIMIC-III, a freely accessible critical care database) through the MIMIC-Extract pipeline. The tasks are in-hospital mortality and vasopressor and ventilation onset, with real missingness. We add the MSKCC breast cancer cohort used in the imputation comparison study, and UCI Heart Disease, Breast Cancer, Default Credit Card, and Ionosphere for controlled stress tests. On the controlled datasets we downsample positives to 20%, 5%, and 1% prevalence. We impose missingness at rates from 10% to 70% under three regimes: completely random, dependent on observed values, and dependent on the outcome. In the outcome-dependent regime, positives are made either more or less likely to be missing, so that the expected sign of the effect is known in advance. Every imputer is fit inside training folds, and test rows are always imputed without labels.
The metrics are AUPRC, recall at a fixed precision, Brier score, calibration slope among predicted-high-risk patients, and the per-patient decision-flip rate across imputations. We pre-register three decisive checks. First, the measured train–test imputation gap must grow with rarity and with minority missingness, and must correlate with the AUPRC deficit of label-including MICE. Second, class-hypothesis imputation must shrink that gap and beat joint label imputation at matched $M$, with the advantage shrinking as $M$ grows, as the variance argument predicts. Third, on MIMIC-III, patients with high $\Delta$ whose missing lab is recorded later should show larger risk revisions than low-$\Delta$ patients. Failure of the first check refutes the mechanism. Failure of the second with success of the first means the remedy is larger $M$, not our method.
References
- Imputation of missing data with class imbalance using conditional generative adversarial networks
- Oversampling and imputation for imbalanced missing data
- MICE and ADASYN for Missing Data Imputation and Imbalanced Data Handling on Heart Disease Classification
- Clinical predictive modelling of post-surgical recovery in individuals with cervical radiculopathy: a machine learning approach
- Comparison of Imputation Strategies for Incomplete Electronic Health Data
- MatchThem: : Matching and Weighting after Multiple Imputation
- Prediction-based Inference in Electronic Health Record (EHR)-linked Biobanks
- Correlation Visualization Under Missing Values: A Comparison Between Imputation and Direct Parameter Estimation Methods
- Machine learning vs. traditional regression analysis for fluid overload prediction in the ICU
- MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III
- MIMIC-III, a freely accessible critical care database