Forecasting Which Organ Fails Next: Ordinal Multi-Organ Early Warning for Sepsis

Early-warning models for sepsis usually answer one yes-or-no question: will this ICU patient meet the sepsis definition in the next few hours? Clinicians need something more specific. Lungs failing, kidneys failing and blood pressure collapsing call for different treatments, so a single alarm leaves the bedside team without a plan, and this contributes to the alarm fatigue that leads staff to ignore warnings. We propose to replace the binary flag with an hourly forecast of which organ system will deteriorate, by how much, and how soon. Severity here is measured on the SOFA scale (Sequential Organ Failure Assessment, a standard 0–4 score for each of six organ systems). The prediction target becomes "organ o will reach sub-score at least k within h hours." The model is built so that predictions for higher sub-score levels and longer horizons can never fall below predictions for lower levels and shorter horizons.

The central bet concerns class imbalance. The usual worry is that finer labels split an already rare positive class into even rarer pieces, a "granularity tax." We argue the opposite holds for this decomposition. Single-organ, one-point worsening events happen far more often than full sepsis onsets, and a sepsis-onset label is by definition a function of SOFA increases. Trained as nested threshold problems, the frequent mild events supply dense supervision, and the rare severe tiers reuse the same learned features. The prediction is that granular forecasting will improve recall on the rarest, most severe events. It should also match or beat dedicated binary models on their own task once the fine-grained forecasts are summed back into a coarse sepsis alarm.

What exists and what is missing

Hourly sepsis early-warning work treats the target as binary. One line combines a Gaussian-process interpolation layer, which turns irregularly timed measurements into a smooth hourly grid with uncertainty, with a recurrent network (Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier). A follow-up adds missingness indicators and medication events (An Improved Multi-Output Gaussian Process RNN). A later paper swaps the recurrent network for temporal convolutions and compares against nearest-neighbour search under dynamic time warping (Moor et al.). The PhysioNet 2019 winner used path signatures, which summarize the shape of a multichannel trajectory, together with LightGBM (Morrill et al.). That model computes a "PartialSOFA" score, but only as an input feature. Its output stays binary, and 67.5% of its detections fired before the intended six-hour window. Attention-based and autoencoder variants on the same challenge also predict a single flag (Liu et al., Dou et al., Yao et al.), as does SepsisLab (SepsisLab). All of this work supplies our encoders and baselines. None of it predicts organ-level outcomes.

Three papers motivate the move to finer targets. Daily SOFA trajectories predict 30-day mortality much better than a single SOFA difference, with AUROC 0.82 against 0.66 over three days (Palmowski et al.). That analysis runs only after diagnosis and at daily resolution, whereas we forecast hourly and before onset. On COVID wards, defining respiratory deterioration as escalation through tiers of oxygen support gave a useful 24-hour predictor (Youssef et al.). That study covered a single organ, had 137 events, and used no imbalance method. In neonatal pulmonary hypertension, three-level staging was clinically necessary, but its F1 fell from 0.84 in cross-validation to 0.63 on held-out data (Ragnarsdóttir et al.). That drop is a warning that fine-grained targets can be brittle, and we take it as a constraint.

Our sharing mechanism draws on DeepTag. DeepTag pulled embeddings of related rare diagnoses toward a shared parent in order to regularize tail classes, and it abstained when confidence was low. It did this for static clinical notes. We carry the idea of sharing along a hierarchy over to time series. The hierarchy we use is ordinal and temporal, organ × severity × horizon, rather than an ontology tree.

Core Claim

Let $p_{o,k,h}(t)$ be the predicted probability, at hour $t$, that organ $o$'s SOFA sub-score reaches at least $k$ within $h$ hours. Each organ and severity level gets a learned per-hour hazard. The probability for level $k$ and horizon $h$ is obtained by accumulating those hazards over levels at or above $k$ and over the hours up to $h$. This construction makes $p_{o,k,h}$ fall as $k$ rises and grow as $h$ grows, with no extra constraints needed. The hazards come from one shared trajectory encoder through a small head for each organ. The rare event "renal ≥3 within 6h" is therefore estimated with the same features that predict the common event "renal ≥1 within 6h." It is not left to a separate classifier trained on a handful of positives.

The coarse sepsis alarm is derived from the forecasts rather than learned separately: it is the probability that the summed SOFA increase reaches at least 2, computed by sampling joint organ outcomes. The claim is that this nested, shared formulation does three things compared with independent per-organ classifiers and flat weighted softmax. It raises severe-tier recall at a fixed alarm budget. It produces alarms that name the organ that actually deteriorates. And it costs nothing on the binary sepsis benchmark.

Research Program

We first build hourly organ-level labels. MIMIC-III records vasopressor doses, Glasgow Coma Scale, urine output and blood gases, so all six SOFA components can be computed there. Our primary cohort is all adult ICU stays. For direct comparison we also use the Sepsis-3 cohort definition of Moor et al., which has 570 cases and 5,618 controls. PhysioNet 2019 lacks vasopressors and GCS. There we restrict to the four lab- and vital-derived components (respiratory via FiO2/SaO2, coagulation, liver, renal-by-creatinine) and use its multi-hospital split to study transfer between hospitals. AmsterdamUMCdb serves as an external test set.

Cardiovascular SOFA is partly defined by treatment: vasopressor use raises the score. Measuring Model Performance in the Presence of an Intervention shows that treatment distorts apparent performance, so we report treatment-defined and physiology-defined organs separately rather than pooling them.

We instantiate the formulation on two backbones, so that any gain can be credited to the target design and not to a particular network. The first is a Gaussian-process interpolation layer feeding a temporal convolutional network, with missingness indicators following Futoma et al.. The second is a signature-feature gradient-boosted model in which each head is trained on the nested cumulative targets with monotonicity constraints. Training uses a summed binary cross-entropy over all (organ, level, horizon) cells, applied to the cumulative probabilities. The only reweighting is a per-cell inverse prevalence that we tune once on validation data. We add an abstention rule in the spirit of DeepTag: when the organ-level forecast is uncertain, the system issues only the coarse alarm. This lets us measure how often specific organ guidance can be given safely.

Baselines

The naive reference is persistence. It predicts that each organ's sub-score in $h$ hours equals its current value plus that organ's average drift in the training data. This asks how much of the granular signal a clinician gets simply by reading current labs. For the coarse task, the naive reference is NEWS. On the Duke cohort NEWS produced about 2.5 false alarms per true alarm at 0.85 sensitivity, against about 0.5 for the Gaussian-process RNN (Futoma et al.).

The strongest existing coarse methods come from Moor et al.. At a 7-hour horizon on MIMIC-III Sepsis-3 they report AUPRC 0.35 for the Gaussian-process temporal convolutional network and 0.40 for nearest-neighbour search under dynamic time warping, with 0.25 for the earlier recurrent model. On PhysioNet 2019 the strongest is the signature LightGBM, with utility 0.360 on the hidden test set and 0.430 in cross-validation (Morrill et al.). We rerun the temporal convolutional network and signature LightGBM on our own patient-level splits, with the same input variables and the same hyperparameter-search budget as our models, and we report their published numbers alongside. We beat them on their own task if the derived coarse alarm reaches AUPRC at least as high as the rerun binary model within bootstrap confidence intervals, while also producing organ-level forecasts. Since Moor et al.'s best number comes from a method that searches the whole training set at inference, we treat 0.40 as a reference point and not as something we must exceed.

The organ-level baselines are the ablations that isolate the key ingredient. The first trains independent binary classifiers, one per (organ, level, horizon) cell, on the same encoder with focal loss, which removes the nested sharing. The second is a flat class-weighted softmax over sub-score values for each organ. The third forecasts the underlying labs and vitals by regression and then computes SOFA deterministically from the forecasts. The proposal succeeds if, at matched alarm rates, it raises severe-tier (≥3) recall and macro AUPRC over tiers above all three ablations.

Why not just…?

Why not just train separate LightGBM classifiers per organ on signature features? This is the most serious threat. Signature-boosted trees won PhysioNet 2019, and per-organ classifiers are simple to build. Our answer is to make the formulation independent of backbone and run it on a boosted-tree backbone as well. If independent per-organ trees match the nested model on severe-tier recall, we will report that the benefit comes from posing the organ-level task at all and not from parameter sharing. That is still a useful result, because no current early-warning model produces organ-level forecasts. We expect the nested model to win specifically on tiers with very few positives, where independent trees have little to learn from.

Why not just forecast the raw labs and compute SOFA from the forecasts? Lab regression is trained to be accurate on average, and accuracy on average favours the typical mild trajectory. This is the "systematic severity under-prediction" failure noted for skewed data. Thresholding a mean forecast will tend to miss rare jumps. We test this directly with the regression ablation, comparing severe-tier recall at equal alarm rates.

Why not just keep the binary alarm and let clinicians read the current SOFA components? That combination is exactly the persistence baseline paired with a binary model. If persistence names the deteriorating organ as often as our forecasts do, the forecasting adds nothing. The "organ-correct alarm rate" metric below measures this.

Why not just reweight classes or oversample? The file notes that duplicating rare positives tends to overfit, and that static class weights ignore how imbalance shifts over an ICU stay. The heart-sound result, with 88.2% specificity against 70.8% sensitivity (Ryu et al.), illustrates the skew that remains without structure. The focal-loss and weighted-softmax ablations test whether reweighting alone closes the gap.

Proposed Experiments

For organ-level forecasts we report per-organ, per-tier AUPRC at horizons of 2, 4 and 6 hours, plus severe-tier recall at a fixed budget of alarms per patient-day. We define the organ-correct alarm rate as the fraction of alarms whose named organ is the one that actually deteriorates. Coverage under abstention is reported alongside it. For the coarse task we report AUPRC on the Moor cohort, PhysioNet utility, and false alarms per true alarm at 0.80 and 0.85 sensitivity.

The decisive checks are three. First, the derived coarse alarm must not fall below the rerun binary models. If it does, the granularity tax is real and we will say so. Second, severe-tier recall must exceed the independent-head and regression ablations on MIMIC-III and must hold up on AmsterdamUMCdb. The pulmonary hypertension staging results warn of collapse on held-out data, so we report internal and external performance side by side. Third, the organ-correct alarm rate must clearly exceed persistence. Results on treatment-defined cardiovascular SOFA will be reported apart from the other organs, as an explicit caveat about intervention bias.