When and how often a patient's labs and vitals are measured is one of the strongest signals in intensive care data. Models that simply count events per hour beat curated severity scores by a wide margin, because a clinician who is worried orders more tests. The same timing also records how a particular hospital operates: batched morning lab rounds, nurse staffing ratios, shift changes, and monitoring hardware that samples every two minutes in one country and hourly in another. Standard pipelines either average the timing away, which is a form of temporal information loss, or sum it up, which mixes the concern signal with the routine signal. The second choice is a plausible, largely untested reason why ICU models degrade when moved between hospitals, a degradation that fragmentation across institutions makes routine.
We propose to split the timing of each measurement stream into two parts. The first is an expected rate that a given hospital would produce for any patient in the same unit, at the same hour of day and day of stay. The second is the patient's excess over that expected rate. The predictive model receives the excess, computed at sub-hourly resolution and counted in orders rather than individual results, instead of raw counts. The claim is testable. If routine drives the transfer gap, the excess representation should keep most of the within-hospital benefit of counting and lose much less performance when a model trained at one hospital is tested at another. If the gap has other causes, a values-only model will transfer just as well, and the idea fails.
Core Claim
Measurement timing is informative but confounded by site. A representation that removes each site's routine sampling pattern should close a substantial part of the cross-site performance gap without giving up the in-site accuracy that count-based models achieve. This framing also resolves an open disagreement in the literature. Temporal-Clustering Invariance in Irregular Healthcare Time Series argues that models should ignore whether tests arrive together or spread apart, treating that as workflow jitter. It coarsens time into bins, appends a count, and ensembles over several resolutions. Nonstationary Gaussian Process Regression for Evaluating Repeated Clinical Laboratory Tests takes the opposite view: it fits a time-warping volatility function to single lab analytes and shows that sampling tracks physiological volatility, and that clinicians undersample volatile periods. Both are partly right. Our decomposition keeps the invariance where it belongs, to routine and to within-panel jitter, and keeps sensitivity where it belongs, to extra orders beyond routine.
Relation to Prior Work
Two companion papers supply the strongest evidence that timing matters and the clearest statement of the risk. Dynamic survival prediction in intensive care units from heterogeneous time series without the need for variable selection or curation and Impact of novel aggregation methods for flexible, time-sensitive EHR prediction without variable selection or cleaning convert every MIMIC-III chart and lab event into a discrete token. They aggregate the tokens in each hour by summation, averaging, or masked softmax, and find that summation wins because it preserves event counts. The same authors report shortcut predictors, such as a priest's visit, and name "counting bias" from staffing as a vulnerability. We build directly on their token pipeline and target that vulnerability, which they identify but do not test across sites.
Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML harmonizes MIMIC-III/IV, eICU, HiRID, and AUMCdb. It documents a transfer gap and finds that preprocessing choices matter more than architecture. It supplies our evaluation setting and a constraint: YAIB's hourly grids already erase timing, so we must return to raw timestamps. MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III supplies intervention-onset labels and the gap-time discipline that prevents outcome leakage.
The temporal-loss literature explains why we operate at sub-hourly resolution rather than recovering detail after pooling. Exploring Temporal Preservation Networks for Precise Temporal Action Localization shows, for video action localization, that deconvolution cannot restore resolution once pooling has destroyed it. CPDA: Class-Conditional Path Distribution Alignment for Unsupervised Time-Series Domain Adaptation aligns class-conditional latent trajectories across domains using pseudo-labels on unlabeled target data. It is the natural adaptation competitor, but it corrects the shift after the fact, whereas we remove a named source of shift at the input.
Finally, Access to care improves EHR reliability and clinical risk prediction model performance is an outpatient analogue that motivates the work. In the All of Us cohort it shows that EHR absence is driven by access barriers, and that an access flag alone does not fix the resulting sensitivity gap. Our ICU work is the tractable, high-resolution case of the same confound between absence of data and absence of disease.
Research Program
We rebuild event streams from raw timestamps in all five YAIB databases, using harmonized concept mappings and keeping each event's exact time and its order identifier. Results that come from one lab panel order are grouped into a single order event. This implements the jitter invariance argued for in the temporal-clustering paper, without coarsening true repeat orders.
For each site $s$ and measurement concept $v$, we fit a routine-rate model $\Lambda_{s,v}(t)$. It is a Poisson regression, or a small boosted count model, predicting the number of orders in 15-minute bins. Its inputs are allowed to describe workflow only: hour of day, day of week, time since ICU admission, ICU unit type, and admission source. It is never given physiological values or outcomes. It is fit on a held-out split of each site's patients so that no patient's own data defines their baseline. For target sites, the model needs only unlabeled timestamps, which is a realistic requirement for deployment. The patient-level input is then the excess
$$e_{v}(t) = \log \frac{N_{v}(t) + 1}{\Lambda_{s,v}(t) + 1},$$
where $N_v(t)$ is the patient's observed order count for concept $v$ in bin $t$. Values enter separately as percentile tokens, following the Deasy et al. pipeline. The downstream sequence model, a GRU over 15-minute steps, is otherwise identical across conditions. Differences can therefore be attributed to how timing is represented, not to architecture.
The main risk is over-removal. If ICU unit type or admission source proxies for case-mix, the routine model absorbs real concern. We therefore report an input-ladder analysis that progressively enlarges the covariate set of $\Lambda$ and tracks in-site accuracy alongside transfer. We expect a peak at workflow-only covariates.
Baselines
The naive reference point is a values-only model. It uses the same tokens and GRU with timing removed: forward-filled values plus a binary observed mask, the standard hourly-grid treatment. It tells us how much timing contributes at all, both in-site and cross-site.
The strongest in-site competitor is raw-count summation aggregation from Impact of novel aggregation methods for flexible, time-sensitive EHR prediction without variable selection or cleaning. That paper reports AUROC 0.878 for 48-hour mortality on MIMIC-III. We rerun it at both hourly and 15-minute resolution on our cohorts, with an identical tuning budget: the same random search size over learning rate, hidden size, and dropout, with selection on source-site validation only. Beating it means matching its in-site AUROC within its confidence interval while losing significantly less AUROC and calibration in leave-one-site-out transfer.
The strongest tabular competitor is LightGBM over 24–48-hour summary statistics that include per-concept counts. Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML reports mortality AUROC of 0.84–0.87 with such models clustering with deep models. We also give LightGBM the excess features, so that the representation question is tested separately from the choice of model.
We include the multiresolution ensemble with grid-and-count from Temporal-Clustering Invariance in Irregular Healthcare Time Series, which reports mortality mAP rising from 51.53% to 53.92% on MIMIC-III. It is the strongest published attempt to regularize away workflow timing.
For adaptation, CPDA is run with target-site unlabeled data. It is a fair competitor because our method also uses unlabeled target timestamps. No reported result was found on Lacuna for CPDA or any trajectory-alignment method on ICU data.
The key ablations are per-site z-scored counts, which remove site means without context-specific routine, and excess computed with a pooled all-site $\Lambda$. The second ablation tests whether site-specificity is the working ingredient.
Why not just…?
Why not just drop timing and use values only? This is a real threat. If values-only transfers as well as our method and nearly matches it in-site, timing is not worth modeling across hospitals, and we will report that. Summation's in-site margin over averaging in the Deasy et al. work suggests timing carries signal worth keeping. The values-only baseline in every transfer experiment settles whether that signal survives a move between sites.
Why not just normalize counts per site? Per-site standardization is the cheapest version of our idea and may capture most of the gain. It is included as an ablation. Our method has to beat it, specifically at sites where routine varies by hour and unit, such as morning lab batching. Otherwise the conclusion is that simple per-site normalization suffices, which would itself be a useful and deployable finding.
Why not just fine-tune on the target site or align domains with CPDA? Fine-tuning needs target labels, which a newly deployed site lacks. CPDA needs no labels, but its own authors note that it amplifies errors when initial pseudo-labels are poor, which is likely under large ICU shifts. We run both, fine-tuning as an upper bound given varying numbers of target labels. We also test whether the excess representation is complementary to CPDA rather than competing with it.
Why not just use LightGBM, since YAIB found architectures tie? YAIB's finding is exactly why we vary the representation and not the model. Excess features are fed to LightGBM as well, so a gain that appears only with the GRU would count against our claim.
Proposed Experiments
The main experiment is leave-one-site-out training and testing across MIMIC-IV, eICU, HiRID, and AUMCdb. Tasks are 24-hour-onward in-hospital mortality and dynamic decompensation. On MIMIC we add vasopressor and ventilation onset with the MIMIC-Extract gap times, where MIMIC-Extract reports random-forest macro AUROC of 0.90–0.94. Metrics are AUROC, AUPRC, expected calibration error, and the transfer gap (source-test minus target-test AUROC), with patient-level bootstrap intervals. No reported cross-site AUROC figure was found on Lacuna to anchor the size of the gap, so we measure it ourselves. Within eICU, we repeat the analysis across individual hospitals, which gives dozens of sites and enough of them to regress each method's transfer gap on the difference in routine sampling rates between source and target.
The decisive check is a workflow-perturbation test. For held-out patients, we resample only the routine component of their order times from another site's $\Lambda$, keeping excess orders and all values fixed, and measure how much predictions shift. A model that encodes concern and not routine should barely move. A raw-count model should move substantially.
The reverse perturbation tests that concern is retained. We inject synthetic extra orders into stable patients. Predictions from our method should rise, and those from the invariance-regularized ensemble should not.
The idea is refuted if the transfer gap for values-only or per-site z-scores matches ours, or if in-site AUROC falls clearly below the summation baseline.
References
- Temporal-Clustering Invariance in Irregular Healthcare Time Series
- Nonstationary Gaussian Process Regression for Evaluating Repeated Clinical Laboratory Tests
- Dynamic survival prediction in intensive care units from heterogeneous time series without the need for variable selection or curation
- Impact of novel aggregation methods for flexible, time-sensitive EHR prediction without variable selection or cleaning
- Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML
- MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III
- Exploring Temporal Preservation Networks for Precise Temporal Action Localization
- CPDA: Class-Conditional Path Distribution Alignment for Unsupervised Time-Series Domain Adaptation
- Access to care improves EHR reliability and clinical risk prediction model performance