A bedside risk model has to state a patient's risk now, using only the data recorded so far. Models allowed to read the whole ICU stay forward and backward do better. BRITS reaches a mortality AUC of 0.850 on PhysioNet 2012 by running recurrent passes in both directions over the full record. Those models cannot run at the bedside, because the backward pass needs data that has not been measured yet. This note proposes a different use for such a bidirectional model. It would not be deployed at all. Instead, it would serve as a training-time hindsight teacher whose risk estimates become the training targets for a strictly causal model, one that sees only the past. For the causal model we use a stack of dilated causal convolutions with residual shortcuts. Each layer adds a small correction to its input, and the network looks back over a long but fixed window. That student issues a risk score every hour with zero added delay. Its output is anchored to a baseline risk computed from the patient's chronic comorbidities, and the temporal network learns only how the current physiology moves risk away from that baseline.
The claim rests on a simple fact: distilling from a hindsight teacher does not bias the causal target, it only denoises it. The claim can be tested with public ICU data and existing baselines. It also gives a principled answer to a question the latency literature usually handles by instinct: how much of the benefit of future context can a causal model recover, and for which patients? We expect the answer to depend on comorbidity. Patients with liver disease or metastatic cancer follow the steep, divergent courses documented in Identifying and analyzing sepsis states. These are small groups whose outcomes are noisy, and that is where denoised targets should matter most.
Core Claim
Let $x_{\le t}$ be a patient's observations up to hour $t$, $y$ the eventual outcome (for example, death or multi-organ dysfunction within the next 24 hours), and $q_T = p(y \mid x_{\le T})$ the calibrated risk estimate of a bidirectional teacher that has seen the full window up to $T > t$. Consider training a causal student $f$ by regression onto the teacher rather than onto the raw binary label. By the tower property of conditional expectation,
$$\arg\min_f \; \mathbb{E}\big[(f(x_{\le t}) - q_T)^2\big] = \mathbb{E}[q_T \mid x_{\le t}] = p(y \mid x_{\le t}),$$
so the student aims at exactly the same causal quantity it would reach by training on labels. The target, however, has variance no larger than that of $y$, since $\mathrm{Var}(q_T \mid x_{\le t}) \le \mathrm{Var}(y \mid x_{\le t})$. Future context is therefore used as privileged information during training and costs nothing at inference. Two quantities decide how large the gain is. The first is how much the teacher's hindsight estimate reduces label noise. That reduction should grow as outcomes get rarer and the prediction horizon longer. The second is how small the patient subgroup is. The equation also bounds what distillation can achieve: it cannot give the student information that only exists in the future. It can only help the student estimate $p(y \mid x_{\le t})$ from fewer effective samples. Any honest evaluation has to separate these two effects, and the experiments below are built to do so.
The student's output is structured as a residual on the risk scale. The logit is $g(c) + r_\theta(x_{\le t}, c)$, where $g$ is a comorbidity model fit on the patient's chronic diagnoses and $r_\theta$ is the causal temporal residual network. The residual network is also conditioned on the comorbidities $c$. The decomposition is readable at the bedside: the model reports how much risk is chronic baseline and how much is acute deviation. The file supports the premise that the baseline carries a lot of signal. Secondary diagnoses outrank the primary admission reason in 30-day readmission modeling in diabetic patients. Comorbidity burden and pre-fracture function dominate outcomes in frail hip-fracture patients.
Relation to Prior Work
Three papers supply the pieces, and none of them combines them. BRITS (BRITS) supplies the teacher. It trains forward and backward recurrent networks jointly, with a consistency loss between their imputations, but it requires the closed sequence and so cannot run online. Interpolation-Prediction Networks (Modeling Irregularly Sampled Clinical Time Series) split each variable into a smooth trend, a non-smooth residual and an observation-intensity channel before a GRU, reaching 0.853 AUC on MIMIC-III. That work is also offline, and its interpolation kernels use observations on both sides of each time point. We borrow its input channels but compute them causally. Dilated causal residual convolutions (Convolutional Sequence Modeling Revisited) supply the student architecture. That work showed they retain memory over longer spans than LSTMs, and it also noted their weakness at inference: they must buffer a receptive field of raw inputs. On hourly ICU data, a one-week receptive field costs 168 stored rows, so the weakness does not bite here.
On the comorbidity side, the MMDL model in Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets already concatenates static features with a GRU over vitals. Adding comorbidities as inputs is therefore not new, and we treat MMDL-style concatenation as a baseline rather than a contribution. What is new is the combination of hindsight-teacher targets with a comorbidity-anchored residual, and the evaluation of both against an explicit latency axis. The latency literature is the source of that axis. Truncated bidirectional recurrent networks (BTRNN) limit future context to a small window, 13 frames in speech denoising. The wait-k work (Robust and Random Wait-k-Tokens) documents the drift that appears when models trained with future context are run incrementally. That drift is a constraint on our design: the student never sees teacher-imputed inputs, only teacher outputs as targets, so no future-derived value ever enters its input stream. Finally, the xLSTM-SENet result (xLSTM-SENet) motivates the question. There, bidirectional processing raised speech quality (PESQ) from 3.26 to 3.48. Whether a gap of that size exists for risk estimation, rather than signal reconstruction, is unknown.
One point needs stating plainly because it reshapes the question. For forecasting a future event, giving a model $k$ hours of look-ahead at decision time $t$ is the same as running a causal model at time $t+k$. In this setting, "bidirectional latency" is simply a delayed alarm. A bidirectional model can therefore be worth more than waiting only through denoising and imputation of past values. It cannot be worth more by knowing the future, since waiting would provide that knowledge too. Our latency curves are designed around this equivalence.
Research Program
We will build four components. The first is a hindsight teacher: BRITS-style bidirectional recurrence over the full stay, with isotonic recalibration on held-out data so that $q_T$ is a calibrated probability, because the tower-property argument requires calibration. The second is a teacher with truncated look-ahead $k \in \{0, 1, 3, 6, 12\}$ hours, in the style of BTRNN, used to trace the latency curve. The third is the causal student: a dilated causal residual convolutional network whose inputs are the causally computed trend, residual and intensity channels plus missingness masks, with output $g(c) + r_\theta$. The fourth is a comorbidity model $g$ fit as regularized logistic regression on the chronic-comorbidity indicators. The loss mixes the teacher-target regression with the ordinary label cross-entropy, and the mixing weight is chosen on validation data.
The analysis has three parts. We will trace AUROC and AUPRC at each decision hour as a function of look-ahead for three model families: the teacher, the causal model evaluated at $t+k$ (the "just wait" curve), and the distilled student at $k=0$. We will measure the variance of teacher targets against that of labels, and relate the per-stratum distillation gain to per-stratum sample size and event rate. The tower-property argument predicts larger gains for small, rare-event strata. We will also check whether the decomposition into chronic and acute risk is stable across random seeds and whether it agrees with the comorbidity-to-trajectory links reported by Fang et al..
Baselines
The naive reference is the comorbidity model $g(c)$ alone, alongside SAPS-II. On MIMIC-III, Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets reports SAPS-II mortality AUROC of about 0.80. We compute both on our own cohorts and decision hours. Any temporal model that fails to clearly beat $g(c)$ at early hours is not contributing acute information.
The strongest offline method on PhysioNet 2012 is BRITS, with a reported in-hospital mortality AUC of 0.850 (BRITS). On MIMIC-III 48-hour mortality, the strongest is Interpolation-Prediction Networks, with AUC 0.853 and AUPRC 0.418 (Modeling Irregularly Sampled Clinical Time Series). MMDL reports 0.9410 AUROC on a different 135-feature MIMIC-III cohort. That cohort is not comparable to ours, so we will rerun MMDL on our split rather than quote its number. These offline models are ceilings, not competitors, since they see the full window. The student "beats" them in a meaningful sense only if it matches their end-of-window AUC while issuing risk scores hourly.
The real competitor is a causal model trained directly on labels, with comorbidities concatenated in the MMDL manner. We run it in two architectures, a causal GRU with decay terms in the style of GRU-D and the same dilated residual network as the student. All models share splits, input channels, parameter count within 10%, and an identical hyperparameter search budget of 50 trials. Beating this baseline means a statistically significant improvement in hour-6 and hour-12 AUPRC, tested with patient-level bootstrap, with no loss in calibration measured by the Brier score.
Two ablations isolate the ingredients. The first removes distillation and keeps the comorbidity anchor. The second removes the anchor and keeps distillation, concatenating $c$ instead.
Why not just…?
Why not just train the causal model on labels with comorbidities as extra inputs? This is the most serious threat. With enough data, label training and distillation converge to the same target, as the equation shows. If MIMIC-III is large enough, the gain may vanish on average. We will test this directly by subsampling the training set to 10%, 30% and 100% and by reporting gains per comorbidity stratum. If distillation helps only at 10% and not in small strata at full size, we will report that it is a data-efficiency tool, not a general improvement.
Why not just wait a few hours, or use a short truncated look-ahead as BTRNN does? Waiting costs time in exactly the patients who deteriorate fastest. The "just wait" curve quantifies this: if the causal model at $t+3$ is no better than the distilled student at $t$, the student has bought three hours for free. If the truncated look-ahead teacher at $k$ beats the causal model at $t+k$, denoising of past values carries real value, and the proposal would add a small-$k$ variant rather than insist on zero delay.
Why not just use static scores and vital-sign thresholds? The file argues against this repeatedly. SAPS-II reaches about 0.80 against 0.9410 for MMDL in the MIMIC benchmark. The TIMI Risk Score reaches 0.68–0.70 against 0.81–0.83 for waveform models in MIL ECG risk stratification. SIRS criteria detected only one of six sepsis states in Fang et al.. We still include $g(c)$ as the floor.
Why not just a causal GRU instead of residual convolutions? On hourly data a GRU may do equally well. The evidence that convolutions win over long sequences, in Convolutional Sequence Modeling Revisited, comes from non-clinical tasks. The architecture is not the core claim, so we run distillation on both students and report whichever is better.
Proposed Experiments and Decisive Checks
We use PhysioNet 2012 (4,000 stays, 35 variables, about 78% missing) to compare directly with BRITS. We use MIMIC-III 48-hour and full-stay cohorts following Shukla et al., with the 30 chronic comorbidities used by Fang et al.. Tasks are in-hospital mortality issued every hour, and 24-hour-ahead progression to the high-mortality organ-dysfunction states identified by Fang. The primary metrics are AUROC, AUPRC, Brier score and alarm lead time at a fixed 20% sensitivity-matched alert rate.
The first decisive check is the latency curve. It shows whether the bidirectional advantage at hour $t$ is larger than the value of simply waiting, which tells us whether any denoising benefit exists to distill. The second is the stratified gain analysis: the proposal predicts larger distillation gains in small, high-event strata such as liver disease and metastatic cancer. A uniform or absent pattern would falsify the variance-reduction explanation. The third is calibration: if the teacher is poorly calibrated, the student's targets become biased and the Brier score should worsen. We will verify this by distilling from an uncalibrated teacher as a negative control. As a shift check in the spirit of TRACER, we train on early MIMIC-III years and test on later ones. The anchored decomposition predicts that when the population shifts, most of the drift falls on $g(c)$, which can be refit cheaply on tabular data without retraining the temporal network.
References
- Identifying and analyzing sepsis states: A retrospective study on patients with sepsis in ICUs
- The 30-days hospital readmission risk in diabetic patients: predictive modeling with machine learning classifiers
- Risk factors for death or serious adverse events within 3-month after surgery in frail older persons with hip fractures and development and validation of a risk prediction model: Based on LASSO-logistic regression
- BRITS: Bidirectional Recurrent Imputation for Time Series
- Modeling Irregularly Sampled Clinical Time Series
- Convolutional Sequence Modeling Revisited
- Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets
- Bidirectional truncated recurrent neural networks for efficient speech denoising
- Training Simultaneous Speech Translation with Robust and Random Wait-k-Tokens Strategy
- xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement
- Multiple Instance Learning for ECG Risk Stratification
- TRACER: Transfer Learning based Real-time Adaptation for Clinical Evolving Risk