When Does a Bedside Forecaster Fail Underrepresented Patients? Time-Since-Admission as the Missing Axis

Hourly ICU deterioration forecasters are trained mostly on majority-group patients. They are then applied to everyone, including patients from underrepresented racial and ancestral groups, and those patients are exactly the ones for whom such models are known to perform worse. Most fairness audits measure that gap at a single moment, usually one admission-level prediction. This proposal argues that the gap has a time structure. A newly admitted patient is a "cold start": there are only a few hours of their own measurements, so the model must lean on what it learned about patients in general, and that knowledge comes mostly from the majority group. As hours of the patient's own history accumulate, their trajectory should increasingly override the population prior. If that is right, the subgroup gap should be largest in the first hours after admission and at the longest forecast horizons, and it should shrink as the stay goes on.

The practical consequence is a specific repair. We start from one model trained on everyone and let a group-specific correction depart from it through a penalty whose strength depends on how many hours of history the patient has, instead of a single fixed strength. The penalty strength is the "borrowing dial". A strong penalty borrows heavily from the everyone-model, and a weak one lets the group correction dominate. We also propose a diagnostic that separates two causes of the gap that are usually conflated. One is too little group data, which is a sparsity problem. The other is that the group's physiology or care genuinely differs, which is a heterogeneity problem. Clinical precedent for the second cause is strong: European-derived BMI thresholds delay monogenic diabetes diagnosis by about ten years in non-White patients (Jones et al.).

Core Claim

We make three claims about hourly mortality and vasopressor-need forecasting on MIMIC-IV.

First, the performance and calibration gap for underrepresented groups is concentrated early in the stay and at long horizons (roughly 8 to 12 hours ahead). An audit that averages over the whole stay hides this.

Second, a sparsity-matched control can tell which cause dominates. We shrink the majority group's training data to the size of each minority group. If that alone reproduces the gap, the gap is statistical cold start. If it does not, the gap reflects real distribution shift.

Third, when shift dominates, borrowing that depends on time since admission beats fixed-strength borrowing. The fixed-strength version is the sequence analogue of CoxKL.

Taken together, the three directions play different roles. Population generalizability defines the failure. The cold-start framing, applied here to a patient's first hours rather than to a new recommender user, gives the mechanism. Forecast latency (how far ahead we predict and how much history we have) gives the axis along which the failure is measured and repaired.

A necessary caveat: MIMIC-IV records self-reported race and ethnicity, not genetic ancestry. We treat these labels as coarse proxies that mix genetic, environmental and care-process differences. We do not claim to identify genetic mechanisms, which the Mendelian randomization literature shows is hard even with genotype data (Zhu et al.).

Relation to Prior Work

Several papers motivate the gap without closing it.

  • Lee et al. show that hourly forecasting with a Transformer and contrastive predictive coding reaches 0.897 average mortality AUROC on MIMIC-IV. Contrastive predictive coding is self-supervised pretraining that learns to predict future hidden states. That work does not break results down by subgroup or by hours of available history.
  • MIMIC-IF audits LSTM and Transformer models on MIMIC-IV across White, Black and Hispanic patients. It finds worse performance for groups with higher baseline mortality, and shorter mechanical ventilation for Black and Hispanic patients. Its predictions are admission-level, not hourly.

We combine these two: the hourly model of the first, audited in the way of the second, along a time axis that neither uses.

Two papers supply the method.

  • CoxKL borrows a European polygenic hazard model into 1,036 African-ancestry men. It penalizes the Kullback-Leibler (KL) divergence from the external model, with one tuning parameter. This raises the C-index to 0.68, compared with 0.49 for the target cohort alone and 0.54 for the external model alone. The authors note it is restricted to static time-to-event outcomes. We carry its penalty over to sequence forecasting and make the dial a function of history length.
  • TAML meta-learns across 6h, 12h and 24h windows for rare clinical events, adapting from 5 to 15 examples. It adapts across time windows but not across patient populations. We keep its multi-horizon view and change what is being adapted to.

Recommender cold-start work constrains the design. MAMO personalizes a meta-learned initialization by indexing memory with static user profiles. Using race as that index would build in exactly the miscalibrated baselines that Jones et al. document. For that reason, our group correction acts on model outputs through a penalty rather than through a profile-keyed initialization. Similarly, DisAlign aligns whole feature distributions across domains, but aligning group representations would erase real physiological differences. We avoid feature alignment for that reason.

Research Program

Base model. We train one pooled hourly forecaster $p_0$ on all MIMIC-IV stays. It follows the Lee et al. recipe: vitals and labs plus chief-complaint text, with contrastive pretraining. It predicts mortality and vasopressor need at horizons of 1 to 12 hours, re-issued each hour of the stay.

Group correction. For each group $g$, we fit a lightweight correction head, giving predictions $p_g$, on that group's training stays. The objective is

$$\mathcal{L}_g=\sum_{i\in g}\sum_{t}\Big[\ell\big(y_{i,t},p_g(x_{i,\le t})\big)+\eta(t)\,\mathrm{KL}\big(p_0(x_{i,\le t})\,\|\,p_g(x_{i,\le t})\big)\Big],$$

The symbols are as follows:

  • $i$ indexes stays and $t$ is hours since admission.
  • $x_{i,\le t}$ is the patient's history up to $t$.
  • $y_{i,t}$ is the outcome label at the chosen horizon.
  • $\ell$ is cross-entropy.
  • $\eta(t)$ is the borrowing dial. It is estimated separately in time-since-admission buckets (0–6h, 6–12h, 12–24h, 24–48h, beyond 48h) by grouped cross-validation.

Setting $\eta(t)$ to a constant recovers a CoxKL-style fixed borrowing rule. Our hypothesis predicts that the fitted $\eta(t)$ rises with $t$: group-specific departures matter most early and fade as the patient's own data accumulates. If the fitted dial is flat, the time-dependence claim fails, even if the gap itself exists.

Baseline-anchoring variant. We also test whether per-patient normalization removes much of the early gap. Here each vital is re-expressed relative to the patient's own first-hours values, which mirrors the lesson from group-specific BMI thresholds.

Sparsity-versus-shift diagnostic. We repeatedly subsample White training stays to match the size of each minority group, retrain, and evaluate on held-out White patients. If this reproduces the minority gap curve over $t$, the gap is attributed to sparsity. If the subsampled majority model still does well, the gap is attributed to shift.

Label-validity check. Vasopressor initiation is itself a treatment decision, and MIMIC-IF documents differential treatment across groups. We therefore report mortality as the primary outcome. We also examine vasopressor results against the timing of recorded physiologic hypotension, so that we do not end up "correcting" toward biased care.

Baselines

Pooled model, no correction. This is the do-nothing reference: the pooled hourly forecaster applied to everyone. Lee et al. report 0.897 average mortality AUROC and 0.897 at the 11–12 hour horizon, with no subgroup numbers. We retrain it on our split so that every method shares the same backbone, features and pretraining. Beating it means a smaller early-stay, long-horizon subgroup gap in AUROC and calibration error, without losing majority-group AUROC.

Group-only model. A model is trained only on each minority group's stays. CoxKL's internal-only result, C-index 0.49, predicts that this fails. Running it here tells us how sparse each group really is in MIMIC-IV.

Pooled model plus group indicator. The group label is added as an input feature. This is the simplest way to let the model learn group offsets.

Pooled model fine-tuned per group. The pooled model is fine-tuned on each group with early stopping, and separately with an L2 penalty toward the pooled weights. This is the strongest practical competitor, because it can learn arbitrary group departures with no new machinery.

Fixed-dial KL borrowing. This is our objective with a constant $\eta$ tuned by the same cross-validation. It isolates the key ingredient: $\eta(t)$ is justified only if it beats this. The closest reported number for this family comes from a different task and dataset: C-index 0.68 on MGI (CoxKL). We found no reported result on Lacuna for KL borrowing in hourly ICU forecasting.

Meta-learning across groups. TAML-style meta-learning treats groups as tasks. On a surgical cohort, TAML reports AUROC 0.809–0.822, against about 0.77 for standard MAML and about 0.75 for a plain deep network (TAML). We found no reported result on Lacuna for TAML on MIMIC-IV subgroups.

Fairness of the comparison. All methods share the backbone, the patient-level splits (stratified by group), the hyperparameter search budget of 50 trials each, and the compute. Results are averaged over five seeds with bootstrap confidence intervals by group.

Why not just…?

Why not just add race as an input feature? The pooled network can then learn group offsets. The catch is that offsets learned from few minority stays are noisy, and a single offset cannot vary with history length unless the model happens to learn that interaction. The indicator baseline tests this directly. If it matches $\eta(t)$ in the early buckets, we will say so, and the contribution reduces to the diagnostic. There is also a deployment objection: MIMIC-IF reports that models already weight ethnicity unevenly across groups, and an explicit race input invites clinical and legal objections that a correction on outputs partly avoids.

Why not just fine-tune the pooled model on each group? This is the real threat. With early stopping, fine-tuning already behaves like implicit borrowing. The difference is that our penalty makes borrowing strength explicit and resolved by hour of stay, so its fitted shape is itself the scientific result. If fine-tuning with L2 regularization matches our method's early-stay calibration within confidence intervals, we will report the method as unnecessary. We would then keep the finding about gap structure and the sparsity-versus-shift decomposition.

Why not just forecast at 24-hour resolution, where data are denser and less noisy? Lee et al. show that coarse 24–48 hour windows lose the precision needed for acute interventions. Fritz et al. note that aggregation smooths away rapid shifts. Coarsening also cannot help the first-hours cold start, because there is no 24-hour window yet. We nonetheless include a 24-hour-aggregated variant and measure its warning lead time at a fixed alarm rate. If it closes the subgroup gap at little cost in lead time, that is a legitimate simpler answer.

Why not just meta-learn across groups with MAML or TAML? With only a handful of groups, there are too few tasks for meta-learning to estimate a good shared initialization. MAMO's evidence against a single global initialization cuts the same way. The TAML-across-groups baseline settles this empirically.

Proposed Experiments

Datasets. The primary dataset is MIMIC-IV with MIMIC-ED chief complaints, restricted to adult ICU stays and split at the patient level. Groups are White, Black, Hispanic and Asian patients, as recorded. Groups too small for stable estimates will be reported as such rather than dropped. External validity is tested by training on MIMIC-IV and evaluating on eICU, the multi-hospital benchmark used by TAML. There, both the group mix and hospital practice differ.

Metrics. For each group, time-since-admission bucket and horizon, we report:

  • AUROC and AUPRC;
  • expected calibration error;
  • warning lead time, meaning median hours of warning before the event at a fixed alarm rate of 10% of patient-hours. This is how forecast latency becomes a measurable clinical quantity.

We do not compare against MIMIC-IF's 0.955 AUROC, because that figure is for admission-level classification, not hourly forecasting.

First decisive check: does the gap have time structure? We plot the pooled model's minority-minus-White gap in AUROC and calibration error against hours since admission and forecast horizon. The core premise fails if the gap is flat in $t$.

Second decisive check: sparsity or shift? We run the subsampled-majority diagnostic. It tells us whether data collection (more minority stays) or modeling (group departures) is the right lever.

Third decisive check: does time-dependent borrowing beat fixed borrowing? We compare $\eta(t)$ against constant $\eta$ and against fine-tuning, measuring early-stay calibration error and lead time for minority groups. The success criterion has three parts. The early-bucket gap must shrink by a margin outside the bootstrap interval. Majority-group AUROC must not fall by more than 0.005. The ordering must hold again on eICU.