Paired-Control Stress Tests for Predicting Where Clinical Models Will Fail

Clinical prediction models often look excellent at the hospital where they were built and then lose accuracy at the next hospital. An ICU mortality model trained on the multi-hospital eICU database fell from an AUC of 0.912 internally to 0.855 on MIMIC-III (A Clinically Practical and Interpretable Deep Model for ICU Mortality Prediction with External Validation). A stroke-bleeding model fell to an AUC of 0.50 on a German outpatient database that lacked inpatient records (Development and validation of a prognostic model predicting symptomatic hemorrhagic transformation in acute ischemic stroke at scale in the OHDSI network). Today these failures are found only after target-site outcomes have been collected, and a single aggregate score says nothing about why the model failed. We propose to borrow a design principle from behavioral neuroscience. Animal researchers never interpret one behavioral test alone. They pair each test with a control test that separates competing explanations. A rat that stops swimming might be despairing or might just be tired, so the swim test is paired with a measure of spontaneous movement.

The proposal is a pre-deployment "stress-test battery" for clinical models built on the same principle. Each test perturbs one family of input features on development-site data. The perturbation changes either the clinical values (what was measured) or the recording process (when and whether something was measured), and it is always paired with a control perturbation that should not matter clinically. The resulting profile shows which feature families the model actually relies on. Combined with unlabeled data from a target hospital, the profile predicts how much the model will degrade there, and on which feature families. The claim is testable: across dozens of hospital-to-hospital transfers, the battery's predicted degradation should rank realized degradation better than internal accuracy, and better than generic shift detectors that ignore what the model uses.

Core Claim

A model fails at a new site when two things coincide: it relies on a feature family, and that family is distributed differently at the new site. Existing tools measure only one side of this product. External validation studies report aggregate AUROC and cannot localize failures (Developing and validating COVID-19 adverse outcome risk prediction models from a bi-national European cohort of 5594 patients, where mortality AUC dropped from about 0.90 to 0.74 in UK Biobank). Behavioral testing of clinical models measures reliance but stays at one site. We define the predicted degradation of model $m$ at target site $s$ as

$$\hat{D}(m,s) = \sum_{g} S_m(g)\,\Delta_s(g),$$

where $g$ indexes feature families (for example, lab values, measurement timing and missingness, vitals, admission codes, demographics), $S_m(g)$ is the model's net sensitivity to perturbing family $g$ on development data after subtracting the response to its paired control, and $\Delta_s(g)$ is the size of the distribution shift in family $g$ between the development and target sites, estimated from unlabeled target records. The new element is the paired-control sensitivity term, and the proposal lives or dies on whether $\hat{D}$ predicts realized drops better than either factor alone.

Where the Pieces Come From

Three behavioral-neuroscience studies motivate the design. The T-794 study paired forced-swim immobility with spontaneous motor activity to show that a drug raised motivation rather than just stimulating movement (Possible Therapeutic Effect of T-794...). This is the paired-control logic we copy. In humans, anhedonic participants kept normal physical force on a joystick task but chose hard-effort trials less often (28.88% vs. 36.41%, p = 0.039), and effort choice did not correlate with sweet-taste thresholds even though both tracked self-report (Objective measures of reward sensitivity and motivation in people with high v. low anhedonia). That constrains us: separate tests may measure separate things, so the battery should report a per-family profile rather than collapse early into one robustness score. The Cloth-ears mouse work ran a broad screen and then used targeted physiology to localize a hearing defect. Otoacoustic emissions (a readout of cochlear hair-cell function) were normal while brainstem latencies were prolonged (Analysis of the mouse mutant Cloth‐ears...). We want the same outcome for models: a result that locates the failure in one feature family.

The perturbation method comes from What Do You See in this Patient? Behavioral Testing of Clinical NLP Models. That work swapped demographic tokens in MIMIC-III admission notes and found hidden failures in models with identical AUROC. It worked only on text, only at one site, and never asked whether test results predict failure elsewhere. We move to structured EHR time series, add paired controls, and validate the battery against realized external performance.

Two papers constrain the proposal. First, domain generalization and adaptation methods did not beat plain training on MIMIC-IV temporal shift, and invariance penalties sometimes hurt by discarding predictive signal (Evaluation of domain generalization and adaptation...). We therefore do not try to train invariant models; we diagnose and select. Second, All models are local argues that one-off external validation is insufficient and that local recurrent monitoring is needed, citing the Epic Sepsis Model's drop from 0.73–0.83 to 0.60–0.64. We accept this. The battery is meant to decide which sites to check first and which candidate model to take into a silent trial. The paper also names an open problem that the battery addresses: it provides no measure of population similarity for deciding when retraining should be triggered.

The translation-gap literature explains why a diagnostic profile matters, not just a prediction. DEPLOYR found that live prospective AUROC fell below retrospective values in nearly every one of 12 lab models, and attributed the drops to feature timing and archiving differences. Those are exactly the "process" features the battery isolates.

Research Program

We will build the battery as a PyHealth-compatible module (PyHealth 2.0) for structured ICU time series. For each feature family it applies three kinds of perturbation. A value perturbation changes measured values while keeping timestamps and missingness fixed. A process perturbation changes measurement frequency, timing jitter, or missingness patterns while keeping values fixed. A null control applies changes that should not alter clinical risk, such as unit-convention re-expression, reordering within a short window, or removing a lab that is routinely absent at the target site.

We will also include positive controls, which are clinically meaningful worsening such as rising lactate or falling blood pressure, where risk should rise. Like confirming that a rat can swim, they identify models that are simply unresponsive. The net sensitivity $S_m(g)$ is the mean absolute change in predicted risk under the target perturbation minus the change under its matched null control, on development data only. The shift term $\Delta_s(g)$ is estimated per family from unlabeled target data using per-family domain-classifier AUC, plus the rate of missingness for features absent at the target.

The testbed is eICU, whose many hospitals allow many transfers. We train on a pool of development hospitals and treat each held-out hospital with enough outcomes as a separate target. MIMIC-III and MIMIC-IV serve as cross-system targets after feature harmonization, and the four MIMIC-IV three-year epochs used by the domain-generalization study give a temporal-shift setting. Tasks are in-hospital mortality, sepsis, prolonged length of stay, and mechanical ventilation. The model zoo covers logistic regression, gradient boosting, the two-level attention LSTM, and a transformer baseline, which yields many (model, site) pairs with realized AUROC and calibration drops.

Baselines

The naive reference is internal performance. It predicts that a model degrades uniformly and ranks models by development-site AUROC. For the eICU-to-MIMIC-III transfer, the strongest reported number is the two-level attention LSTM's external AUC of 0.855, against 0.912 internally and 0.741 for logistic regression externally (A Clinically Practical and Interpretable Deep Model...). We will reproduce this transfer as a sanity check before trusting our pipeline.

The strongest competitor for predicting degradation without target labels is a set of data-centric shift detectors and label-free performance estimators. These include a whole-record domain classifier, importance-weighted estimates of target AUROC, and confidence-based accuracy estimates. We found no reported result on Lacuna for label-free performance estimation on eICU or MIMIC transfers, so we will run these methods ourselves. They get the same unlabeled target data, the same development models, and a matched tuning budget chosen on a separate set of calibration sites.

A second strong competitor is attribution-weighted shift: SHAP importance per family multiplied by $\Delta_s(g)$. It tests whether ordinary feature importance can stand in for the paired-control battery. The key ablations are sensitivity without null-control subtraction, the value-only and process-only halves of the battery, and $\Delta_s$ alone.

The proposal beats these baselines if $\hat{D}$ achieves a higher Spearman correlation with realized AUROC drop across held-out (model, site) pairs than every baseline, with the gap confirmed by a bootstrap over sites. It must also reduce selection regret, meaning the target-AUROC shortfall relative to the best model in the zoo, when used to pick one model per site. If attribution-weighted shift matches the battery, the paired controls are not earning their cost, and we will report that.

Why not just…?

Why not just do local validation at every site, as "all models are local" recommends? When target labels exist, measured performance beats any prediction, and we do not claim otherwise. But labels arrive late: mortality and sepsis outcomes take days to weeks. Silent trials such as DEPLOYR are expensive to run at one site, let alone many. And local validation tells you that a model failed, not which inputs caused it. The battery's role is triage and model choice before labels arrive. Selection regret against a label-based oracle measures exactly how much local labeling it can replace.

Why not just use a data-shift detector on the target site? This is the most serious threat. Shift detectors flag sites that differ, but many differences involve features the model ignores. The OHDSI German collapse came from missing inpatient features the model depended on, not from shift in general. The $\Delta_s$-only ablation isolates this. If it performs as well as $\hat{D}$, model-side testing adds nothing and we will say so.

Why not just train on more hospitals, or with an invariance penalty? Broad-data training is a good default, and the domain-generalization study found that invariance penalties did not beat it (Evaluation of domain generalization and adaptation...). But pooling does not tell a deploying hospital whether its own workflow falls outside the pool. We include pooled-training models in the zoo, and we test whether the battery still predicts which held-out sites degrade them.

Why not just read feature importance? Importance scores computed on development data confound clinical signal with recording artifacts. A lab's timestamp density can carry "importance" because sicker patients are measured more often. The value-versus-process split with null controls is designed to separate these, the way the swim test is separated from general motor activity. The SHAP-weighted baseline decides whether that separation matters in practice.

Decisive Checks

The first check is predictive validity: across held-out eICU hospitals, the MIMIC-III and MIMIC-IV cross-system targets, and MIMIC-IV temporal epochs, $\hat{D}$ must rank realized AUROC and calibration drops better than internal AUROC, shift detectors, and SHAP-weighted shift. The second is localization: for the worst transfers, we retrain with the flagged feature family removed or harmonized. If target performance recovers more than it does after removing an unflagged family of similar size, the battery found the true cause. This mirrors the Cloth-ears step from broad screen to targeted confirmation.

The third check is a replication of known failures. The temporal sepsis decay of 0.090 AUROC reported on MIMIC-IV should appear in the battery as reliance on definition-sensitive process features. A deliberately constructed "missing inpatient records" target should reproduce the OHDSI-style collapse toward 0.50 AUC, and the battery should predict that collapse from the missingness term. If these three checks fail while the attribution or shift-only baselines pass, the paired-control idea does not transfer from animal behavior to clinical models, and we will report that result.