Hospital risk models that update every hour, such as models that track a patient's chance of dying or developing sepsis during an intensive care stay, are routinely validated at a second hospital before anyone trusts them. Fairness audits of these models are almost always run at the hospital where the model was trained, and at a single fixed point, typically "first 48 hours." This note proposes a specific, testable explanation for why subgroup disparities change when such a model moves to a new hospital. The explanation is the measurement process: how often each patient's labs and vitals are recorded. Monitoring frequency differs between hospitals. It also differs between demographic groups within a hospital. A model that learns "frequent lab draws mean danger" at one site carries that habit to a site where frequency means something else. We expect the damage to fall unevenly across groups, and to grow or shrink over the hours of a stay.
The work has two parts. The first is a diagnostic: a frozen model is re-scored on external patients whose observation schedules have been artificially thinned toward the source hospital's pattern, which isolates how much of the transported subgroup gap is explained by monitoring intensity. The second is a training method that needs no labels from the new hospital and does not use race at deployment. It randomly thins observation schedules during training so the model cannot use monitoring intensity as a stand-in for demographic or site identity. Both are evaluated hour by hour, across the harmonized multi-center ICU databases.
Core Claim
The claim is this. For hourly ICU risk models moved between hospitals, a large share of the change in subgroup calibration is explained by subgroup-by-site differences in measurement frequency. Subgroup calibration here means the gap between predicted and observed risk for each group, tracked over hours since admission. If the claim holds, schedule-randomized training should shrink that transported gap at external sites more than standard recalibration does, and the accuracy cost should be small.
The claim is falsifiable in a direct way. If thinning external schedules toward the source pattern does not move the subgroup gap, the hypothesis is wrong, and the proposal falls back to a time-resolved external fairness audit, which is useful but modest.
What Prior Work Leaves Open
Four papers motivate the gap, each from a different side. A Controlled Study of Fairness Interventions for Temporal Graph Transformers on ICU Mortality Prediction shows that the structure of a temporal model changes race disparities on MIMIC-IV mortality: linking time steps by physiological similarity narrows the race AUROC gap (0.129 for the full graph model versus 0.135 for an LSTM), while purely sequential links widen it. That result comes from one hospital and one fixed 48-hour window, and the authors flag transportability as open. Investigating Data Interventions for Subgroup Fairness: An ICU Case Study treats eICU hospitals as separate data sources. It finds that adding external data can widen gaps, and that "mean discrepancy" (predicted mean risk minus observed incidence within a group) is the quantity that matters. Its outcomes are static, so it says nothing about how that discrepancy evolves within a stay. We reuse its mean-discrepancy metric and index it by hour. MIMIC-IF documents treatment differences by ethnicity, such as shorter mechanical ventilation for Black and Hispanic patients, and finds that model attributions differ by group. That is direct evidence that the care process, and with it the observation process, is entangled with demographics. All models are local argues that workflow and protocol drift make one-off external validation unreliable, citing the Epic Sepsis Model falling to an AUC of 0.60–0.64 externally. Its analysis stays at the aggregate level and does not ask who bears the drift.
Two papers supply methods or constrain them. Yet Another ICU Benchmark (YAIB) provides harmonized cohorts and tasks across MIMIC-III/IV, eICU, HiRID and AUMCdb. It also shows that data-preparation choices often matter more than architecture. This is a warning that missingness and imputation handling, the exact lever we study, could dominate results, so our comparisons hold everything else in the pipeline fixed. KFDeep shows that feeding elapsed time between visits into a recurrent model adds 3.0–4.3% AUROC, and that AUROC drops from 0.9311 internally to 0.8141 externally. Its authors do not determine whether the drop comes from demographics, incidence shifts, or visit frequency. Our diagnostic is designed to answer that kind of question for ICU data.
A tension has to be faced directly. Time-gap features help, as KFDeep shows, precisely because measurement timing carries information. Removing that information could therefore cost accuracy. The proposal does not assume that cost is small; it measures it.
Research Program
The first step is to describe the measurement process. On YAIB-harmonized cohorts we estimate, for each site and subgroup, the hourly rate at which each variable is recorded. Lab panels, blood gases and vitals are tracked separately, and rates are conditioned on acuity strata (an admission severity score), so that genuine severity differences are not mistaken for practice differences. The demographic attributes used are age and sex across all databases, and race where it is recorded, chiefly in MIMIC-IV and eICU. We expect race to be sparse or absent in the European databases and will report exactly what is available rather than impute it. The deliverable is a map showing where the same acuity stratum is monitored at different intensities across sites and groups.
The second step is the schedule-thinning diagnostic. Take a model trained at a source site, frozen. For external patients from groups whose monitoring is denser than the source pattern, delete observations at random so that, within each acuity stratum, their schedule rates match the source rates. Then re-score them without any retraining. Observations cannot be added, so where the external site is sparser the direction is reversed: the source training data are thinned, the model is retrained, and it is evaluated on unmodified external data. For each pair of sites, the result is a curve over hours since admission. It shows the subgroup mean-discrepancy gap before and after schedule matching, and the fraction of the transported gap that schedule matching removes.
The third step is schedule-randomized training. During training, each patient's observations are thinned at a rate drawn at random from the range of rates seen across the training sites, independently for each variable family. Missingness indicators and time-since-last-measurement features are recomputed after thinning. The model therefore sees the same physiology under many plausible monitoring regimes and cannot rely on intensity alone. The method needs no target labels and no protected attribute at deployment. It is applied to a GRU, a Transformer from YAIB's model zoo, and LightGBM on hourly rolling features, so that the effect is not tied to one architecture.
Baselines
The naive reference point is a model trained at the source with YAIB's default preprocessing and applied unchanged at the external site. Its hour-indexed subgroup gaps define "doing nothing." No reported result found on Lacuna gives time-resolved, transported subgroup calibration for any ICU model, so this baseline's numbers will be new.
The strongest discrimination baseline is LightGBM on engineered summary features. It is the best aggregate model in the controlled TGT study, at AUROC 0.837 for 48-hour MIMIC-IV mortality (Controlled Study), and YAIB reports mortality AUROCs of 0.84–0.87 across datasets with classical models competitive. On sepsis transport, the reference is the self-attention model of Predicting sepsis in multi-site, multi-national intensive care cohorts: internal AUROC 0.847, external 0.761. LightGBM gets the same hourly features, the same splits, and the same hyperparameter search budget as the neural models.
The strongest fairness baseline that needs no target labels is the hybrid from Investigating Data Interventions. It selects training sources to minimize mean discrepancy, then applies isotonic calibration on source validation data. No reported number found on Lacuna for its subgroup results; the paper reports them qualitatively.
The strongest label-using baseline is local recalibration at the target site. We give it a few hundred labeled target patients and fit isotonic regression separately within each block of hours since admission, without using race. This is the "recurrent local validation" response of All models are local.
For reference only, we also run per-subgroup threshold equalization after Platt scaling. Internally it cut the TPR gap from 0.20–0.23 to below 0.03 (Controlled Study), but it requires race at decision time. It shows how far a legally fraught method gets, not a competitor we must beat.
Two ablations isolate the key ingredient. The first replaces site-matched random thinning with uniform-rate thinning, to test whether matching the actual spread of site rates matters. The second drops missingness indicators and time-gap features entirely, the bluntest way to remove intensity information.
Beating the baselines means two things at held-out external sites, averaged over hours. First, worst-group mean discrepancy and the worst-group-minus-best-group gap are lower than both the naive model and the data-selection hybrid. Second, pooled external AUROC is no more than 0.01 below the unmodified model. Matching local recalibration with zero target labels would count as a strong result.
Why not just…?
Why not just recalibrate locally at each new hospital? This is the real threat. If a few hundred labeled target patients and time-binned isotonic regression close the gap, the training method adds little once labels exist. The experiments trace a label-budget curve (0, 50, 200, 500, 1000 target patients) to find where local recalibration catches up. The proposal's claimed value is the period before labels accumulate, such as silent-mode deployment, plus the diagnostic explanation of why the gap appeared. Recalibration does not provide that explanation. Recalibration also fixes miscalibration, but not ranking errors within a group that come from misread monitoring signals. Subgroup AUROC tests that distinction.
Why not just use LightGBM on static summaries? It matches deep models on aggregate AUROC in the cited benchmarks. However, summary features such as counts and means still encode measurement intensity. The diagnostic and the thinning method apply to it directly, and we run both on LightGBM, so this is a condition in the study rather than an alternative to it. If LightGBM shows no intensity-driven transported gap while sequence models do, that is itself a finding about where the bias enters.
Why not just drop missingness indicators and time gaps? That is our second ablation. KFDeep's 3.0–4.3% gain from time intervals suggests that dropping them costs accuracy. Thinning is intended to keep timing information that reflects physiology while breaking its link to site-specific practice. If the blunt removal performs as well, we will report that and recommend it.
Why not just train on more pooled sites? Investigating Data Interventions shows that naive data addition can widen subgroup gaps. We still include a pooled multi-site model as a condition, because YAIB finds eICU-trained models transfer better. Our hypothesis predicts this is partly because eICU already spans many monitoring regimes, which is a "natural" version of thinning.
Proposed Experiments
We run leave-one-database-out transfer across MIMIC-IV, eICU, HiRID and AUMCdb through YAIB. For race-specific analysis, we use transfers between eICU hospitals, treating individual hospitals as targets, following the data-interventions study. Tasks are YAIB's hourly mortality and sepsis-onset predictions. Metrics are computed in 6-hour blocks up to 72 hours: subgroup mean discrepancy, calibration slope, subgroup AUROC, and TPR at a single global threshold set for 80% overall recall.
There are three decisive checks. The first is the explained fraction: if schedule matching removes less than roughly a quarter of the transported subgroup gap in most site pairs, the measurement-process hypothesis is weak, and the contribution reduces to the time-resolved audit. The second is the time profile: the hypothesis predicts the intensity-driven gap is largest early in the stay, when monitoring is most protocol-driven, and smaller later, when physiology dominates. A flat profile would undercut the mechanism. The third is the label-budget crossover against local recalibration, which determines whether schedule-randomized training is a deployment tool or mainly an explanatory device. Each result is reported per site pair rather than only averaged, because the "all models are local" argument predicts that the answer will vary by hospital.
References
- A Controlled Study of Fairness Interventions for Temporal Graph Transformers on ICU Mortality Prediction
- Investigating Data Interventions for Subgroup Fairness: An ICU Case Study
- MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset
- All models are local: time to replace external validation with recurrent local validation
- Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML
- Development and Validation of a Dynamic Kidney Failure Prediction Model based on Deep Learning: A Real-World Study with External Validation
- Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning