Models that predict ICU mortality from vital signs must choose how far back to look, and every choice costs something. A long lookback, such as the 24 to 48 hours used by current hybrid convolutional-recurrent models, captures a patient's overall trajectory. It also averages away the sudden crash that often signals deterioration. A short lookback sees the crash but misses the trend. We propose training several copies of the same sequence model, each on its own lookback length, and treating the points where they disagree as information rather than noise. When a model that saw the last six hours becomes much more alarmed than a model that saw the last forty-eight, the patient has probably just changed state. That gap should tell the final prediction to trust the short view.
Put in two sentences: build an ensemble whose members differ only in how much history they see, and use the disagreement between short-history and long-history members to decide how to combine them. We claim this recovers acute deteriorations that a single wide-window model smooths over. We also claim it does so better than ensembles whose diversity comes only from random seeds, and better than a single network with several window branches trained jointly.
Core Claim
The prior work pulls in two directions. Evidence from speech shows that one model cannot simply be given a wider window. Toward a Better Understanding of Deep Neural Network Based Acoustic Modelling found that widening input context from ±10 to ±20 frames lowered training loss but did not improve test word error rate. The ICU literature shows the two timescales carry different information. Predicting ICU mortality by supervised bidirectional LSTM networks argues that deterioration shows up as abrupt fluctuations that sequence models smooth over. The convolutional front ends used in hybrid models behave like low-pass filters: the CNN-BLSTM dysarthria study found learned filters centred below 800 Hz. Our claim is that when members are trained independently at different lookback lengths, their prediction gap $d_t = p^{\text{short}}_t - p^{\text{long}}_t$ is a usable signal. Here $p^{\text{short}}_t$ and $p^{\text{long}}_t$ are the mortality probabilities that the shortest-window and longest-window members output at hour $t$. We claim this signal predicts imminent adverse outcomes beyond what the mean prediction captures, and beyond the spread among seed-only members. The fusion rule we test is
$$\hat p_t = \sigma\big(a + b\,\bar z_t + c\,z^{\text{short}}_t\cdot g(|d_t|)\big),$$
where $\bar z_t$ is the mean logit of all members, $z^{\text{short}}_t$ is the short-window member's logit, $g$ is a small monotone function learned on validation data, $a, b, c$ are scalars fit on validation data, and $\sigma$ is the logistic function. The model is a stacking head that is allowed to trust the short view in proportion to how strongly it disagrees with the long view. We keep it deliberately small: the base-model and fusion literature warns that too many fusion inputs add noise (Adaptive expert fusion model).
Relation to Prior Work
Two ICU papers supply the backbone and the reference numbers. AIMS (Continuous and automatic mortality risk prediction using vital signs in the ICU) passes a fixed 24-hour window of vital-sign statistics through two temporal convolution layers and a bidirectional LSTM, which reads the sequence both forwards and backwards. AIMS trains separate networks for the 3-, 7- and 14-day outcome horizons, but it varies the outcome being predicted, never the lookback, and it never combines models. BiLSTM-ST supervises a bidirectional LSTM at every time step. It reports that a 36-hour lookback matches 48 hours, which is further evidence that the lookback is underdetermined. Neither paper ensembles.
Three papers supply ensembling ideas that we reuse or contrast with. PEAMP (Pipelined Ensemble Architecture for Mortality Prediction on MIMIC III) averages a neural network and a random forest over static summaries. It motivates the question of whether sequence modelling is needed at all, and our static baseline answers it. EFM+ (Adaptive expert fusion model) weights heterogeneous experts by their recent local error. That weighting needs labels to arrive online, which ICU mortality cannot provide within a stay. Our gate conditions on disagreement between members, which is available without labels. WWAggr (A Window Wasserstein-based Aggregation for Ensemble Change Point Detection) is the closest method. It calibrates seed-diverse ensemble outputs and compares prediction distributions in adjacent history and future windows to detect change points. WWAggr applies windowing after the members have made their predictions, and every member sees identical inputs. We make lookback length the source of diversity itself, and we turn cross-scale disagreement into a gate for risk estimation rather than a change-point score. We adapt WWAggr as an ablation. Finally, the prediction-error module in Multi-Head CNN-LSTM with Prediction Error Analysis shows that deep sequence stacks need an auxiliary signal to catch sudden changes. That supports our motivation, but our auxiliary signal comes from the ensemble itself rather than from a hand-built statistical forecaster.
Research Program
We fix one backbone, the AIMS-style two-layer temporal convolution followed by a bidirectional LSTM, operating on hourly vital-sign sequences. We then train K = 5 members at lookbacks of 6, 12, 24, 36 and 48 hours. They share the same architecture, data split and tuning budget, and each has its own seed. All members produce predictions hourly, which gives a risk trajectory for each patient. We fit calibration (Beta calibration, following WWAggr) and the fusion head on a validation fold only.
The analysis has two parts. First, we test whether $|d_t|$ has predictive value. We regress outcome on the mean prediction plus $|d_t|$, then compare against the same regression using the standard deviation across five seed-only members at the best single window. Second, we test whether the gated fusion actually improves prediction. We break results down by patient subgroup. Our pre-registered "acute" subgroup contains patients whose final-12-hour heart rate, mean arterial pressure or oxygen saturation deviates from their own earlier mean by more than two within-patient standard deviations. The complementary group is labelled "stable." The hypothesis predicts gains concentrated in the acute subgroup and near-zero change for stable patients. If gains are uniform, we are only seeing generic ensembling.
Baselines
The naive reference is a single AIMS-style CNN-BiLSTM with a 24-hour lookback, trained once. It is the setup a practitioner would ship, and every other number is read relative to it. The strongest reported single model on PhysioNet/CinC 2012 in the file is BiLSTM-ST, at AUC 0.869, F1 0.793 and precision 0.848 (Predicting ICU mortality by supervised bidirectional LSTM networks). On MIMIC-III, AIMS-3 reports AUROC 0.884 for 3-day mortality (AIMS). We will reimplement both on our splits, because published numbers come from different preprocessing. The published figures serve as sanity anchors; our reimplementations are the actual competitors. PEAMP reports about 92% accuracy on MIMIC-III (PEAMP). Accuracy is not comparable under class imbalance, so we rerun PEAMP's neural-network-plus-random-forest averaging on static summaries of each lookback and compare it by AUROC.
The critical competitor is a seed ensemble of five members at the single best lookback, chosen on validation data. It has exactly the same training and inference compute as our method, and it answers whether the diversity comes from the lookbacks or simply from having five models. The second critical competitor is a single multi-branch network: five convolution-LSTM branches at the five lookbacks, trained jointly end to end with a learned gate over the same disagreement feature. Its parameter count and training epochs are matched to the ensemble.
Two ablations isolate the proposal's key ingredient. The first uses our window-diverse members but aggregates them by plain mean and by an unconstrained stacking head without $|d_t|$. The second replaces our gate with a WWAggr-style Wasserstein score between the history and future windows of the pooled predictions. All methods get the same hyperparameter search of 30 configurations per member family, the same folds, and five repeated splits.
We count the proposal as beating a baseline under one of two conditions. The first is that its AUROC on the acute subgroup exceeds that baseline's by a margin larger than the 95% bootstrap interval across repeated splits, with no significant loss on stable patients. The second is that at matched specificity it produces alarms earlier, measured as the median lead time before death at which the alarm first fires.
Why not just…?
Why not just ensemble seeds at the best window? This is the cheapest version of "more models" and a real threat, because seed diversity alone gave large gains in WWAggr and in the DialDoc21 ensemble. Our argument is that seed members share the same low-pass view of the data, so they disagree least on exactly the patients who crash. The matched-compute seed ensemble is the head-to-head test. If it ties our method on the acute subgroup, the lookback-diversity claim fails and we will say so.
Why not just train one network with branches at several window lengths? Multi-scale architectures are standard, and a jointly trained model could in principle learn the same gate. We expect joint training to pull the branches toward agreement, which erases the signal we want to use. This is a hypothesis, not established evidence. The matched multi-branch baseline settles it. If it matches the ensemble at lower inference cost, the practical recommendation becomes the single model, and our contribution narrows to diagnosing why a disagreement gate helps.
Why not just use a wider window, or global self-attention, in one model? On window size, the speech result from Toward a Better Understanding of DNN Acoustic Modelling shows that wider context improved training loss without improving test error. On attention, BiLSTM-ST reports that attention smoothed the ordered local fluctuations its time-weighted loss caught. We include the 48-hour single model in our lookback sweep, and we add a Transformer, a self-attention sequence model, over 48 hours at the same budget, so both claims are tested directly on our data.
Why not just use static summaries with a random forest? PEAMP's strong reported accuracy makes this tempting. However, the file reports sequence models far ahead of simple baselines on PhysioNet 2012: a plain CNN reached AUC 0.717 and BiLSTM-ST reached 0.869. The rerun PEAMP baseline measures this gap on our splits.
Proposed Experiments
The primary benchmark is PhysioNet/CinC 2012 in-hospital mortality, using 48 hours of irregular multivariate records resampled hourly. The second is MIMIC-III, with 3-day mortality predicted hourly from ICU vital signs, following the AIMS setup. Metrics are AUROC and AUPRC overall and per subgroup, expected calibration error, and alarm lead time at 90% specificity. We also report inference cost as the number of forward passes per hour, since cost that grows linearly with ensemble size is the known weakness of ensembles.
As a transfer check outside medicine, we run the same design on C-MAPSS turbofan remaining-useful-life prediction, with lookbacks from 10 to 50 cycles. The reference point there is the multi-head CNN-LSTM with prediction-error analysis, which reports RMSE 12.19 on FD001 (Multi-Head CNN-LSTM with PEA). This asks whether cross-scale disagreement flags onset of degradation the way it would flag acute deterioration.
Three results would decide the question. The first is whether $|d_t|$ adds significant predictive value over both the mean prediction and seed spread. The second is whether gains concentrate in the acute subgroup and not the stable one. The third is whether independent training beats the matched jointly trained multi-branch model. If the first fails, the idea is wrong. If only the third fails, the idea survives as a cheaper single-model design.
References
- Toward a Better Understanding of Deep Neural Network Based Acoustic Modelling: An Empirical Investigation
- Predicting ICU mortality by supervised bidirectional LSTM networks
- Raw Speech Waveform Based Classification of Patients with ALS, Parkinson’s Disease and Healthy Controls Using CNN-BLSTM
- Adaptive expert fusion model for online wind power prediction
- Continuous and automatic mortality risk prediction using vital signs in the intensive care unit: a hybrid neural network approach
- Pipelined Ensemble Architecture for Mortality Prediction on MIMIC III
- WWAggr: A Window Wasserstein-Based Aggregation for Ensemble Change Point Detection
- Multi-Head CNN-LSTM with Prediction Error Analysis for Remaining Useful Life Prediction
- Technical Report on Shared Task in DialDoc21