Bedside early-warning models for sepsis look accurate inside the hospital that trained them and degrade in the next one. In the PhysioNet 2019 challenge, the best teams scored between 0.33 and 0.36 on the challenge's clinical utility score, and performance fell markedly on the hidden third hospital, where the distributions of key variables differed sharply (Early Prediction of Sepsis from Clinical Data). Two other fields have a design that targets this kind of failure. In network security and industrial inspection, a cheap first stage flags anything unusual, and an expensive second stage decides whether the flagged event matters. The second stage only ever sees the small set of flagged cases. We propose to bring this two-stage design to bedside monitoring with two changes. The first stage is calibrated to each patient's own recent vital signs rather than to a population. The second stage reads the trajectory of sparse laboratory biomarkers after the flag, meaning whether lactate, creatinine, or white-cell count is rising, falling, or flat.
The idea in two sentences: a per-patient change detector on hourly vitals fires whenever a patient departs from their own baseline. A second model, trained only on those fired episodes, decides whether the departure is transient and benign or the start of a sustained deterioration, using how the vitals and labs evolve afterward. The bet is that both tasks are more alike across hospitals than the raw lab values a single end-to-end model learns from. "Departed from your own baseline" is defined per patient, and "did it resolve or persist" is a question about the shape of the trajectory. If the bet holds, the two-stage system should lose less utility when moved to a new hospital.
Core Claim
Splitting sepsis warning into a patient-relative tripwire and a second-stage trajectory review reduces the performance drop across hospitals. The gain should come from two specific ingredients, not from any extra modeling capacity. The first is that the tripwire's input is normalized against the patient's own history. The second is that the review model is trained on the conditional population of fired episodes. That population contains "rare but benign" excursions, the analogue of the infrequent-but-legitimate traffic separated out in CADeSH. It also contains episodes that went on to sepsis.
The claim is falsifiable in a specific way. A single end-to-end model given the same patient-relative features, the tripwire's output as an input feature, and the same class reweighting should transfer as well, if the decomposition itself does no work. We treat that model as the decisive control. A null result would be informative in its own right: it would mean the transfer benefit lives in the feature normalization, not in the two-stage architecture.
Where This Sits
Three lines of work motivate the design.
Longitudinal studies show that biomarker direction and speed of change carry information that a single measurement misses. Serial albumin during myocardial infarction shows an S-shaped relation to mortality that admission values miss (albumin changes in AMI). Rising and falling hsCRP trajectories predict metabolic syndrome differently (hsCRP trajectories). Static admission neutrophil-to-lymphocyte ratio misreads evolving critical illness (NLR in critical illness). Other studies document transient, non-pathological spikes, which are exactly the excursions a tripwire will catch and a reviewer must dismiss: exertional D-dimer (D-dimer in T2D), pulsatile insulin (fasting insulin), and white-coat blood pressure (BP in children). These papers supply the clinical premise, not the method.
One study constrains the design. p-Tau217 survival analysis reaches a 4-year conversion AUROC of 0.787 from a single baseline measurement. That is a reminder that trajectories are not always necessary. We therefore include a snapshot-only reviewer as an ablation rather than assuming trajectories help.
The two-stage literature supplies the method.
CADeSH runs an autoencoder gate followed by clustering that separates rare-benign from malicious traffic, reaching an FPR of 0.014. It pools identical devices to define normal. We cannot do that with heterogeneous patients, which is why our gate is per-patient.
VELM gates an industrial image anomaly detector into GPT-4o for defect categorization. It works on static images with closed classes.
Lead-then-local spam detection monitors one designated lead signal with a CUSUM detector (a running sum of deviations that sounds an alarm when it crosses a limit) and then runs local checks on secondary signals. We borrow that tripwire but cannot assume a single lead vital, because sepsis presents diffusely.
Two methods in the file are the closest temporal work, and neither does what we propose.
THOC builds multi-scale nested hyperspheres inside one network on regularly sampled telemetry, reaching an F1 of 95.18% on SMAP. It has no gate and no treatment of sparse, irregular labs.
Metabolic pathway prompting screens and then "rethinks" longitudinal steroid profiles with an LLM. Both of its passes are LLM calls, it relies on athlete panels, and its review is not trained on a gated conditional population.
TFT-multi forecasts five ICU vitals with an observation-masked loss and quantile outputs, reducing Mean BP MAE from 10.1 to 6.2. It is a forecaster, not a detector, and we use it as a candidate source of tripwire residuals.
Research Program
Stage 1: the per-patient tripwire. It runs on hourly vitals and on whatever labs happen to arrive. For patient $i$ and variable $x$, it keeps a running robust baseline $\mu_i$ and scale $\sigma_i$ estimated from that patient's own history. These are initialized from population priors and shrink toward the patient's values as hours accumulate. Each variable gets a two-sided CUSUM statistic:
$$S_t = \max\left(0,\; S_{t-1} + \frac{x_t - \mu_i}{\sigma_i} - k\right),$$
where $k$ is a slack constant that ignores small drifts. An alarm fires when $S_t$ for any variable, or a combined multivariate version, exceeds a limit $h$. We set $h$ to hit a target false-trigger rate per patient-day on non-septic training patients.
We compare this tripwire against a second variant whose input is the residual from TFT-multi's median forecast, with $\sigma_i$ replaced by the width of the forecast's 10–90 percentile band. Only fixed trigger rates are compared, so the comparison is about which signal fires earlier, not about different alarm budgets.
Stage 2: trajectory review. Each trigger opens an episode window. The window contains a short look-back to the pre-trigger baseline and a look-forward that grows hour by hour until the review commits or a time cap is reached. Inside it, the reviewer receives three things:
- patient-relative vital trajectories;
- for each lab drawn in the window, its change from the patient's last pre-trigger value and its slope;
- explicit observation masks, in the spirit of TFT-multi's masked loss, so that lab absence is visible rather than imputed.
The reviewer is a small irregular-time sequence model. We also train a gradient-boosted alternative on hand-built trajectory features. Training data are only the fired episodes from the training hospitals. Positives are episodes followed by Sepsis-3 onset within the challenge horizon. Negatives are triggered episodes that resolved without sepsis.
Analysis of the negatives. This set of hard negatives is the core of the proposal, and we will characterize it directly. We will measure what fraction of resolved episodes look like transient single-variable spikes and what fraction look like multi-variable drifts that recovered. We will cluster the resolved episodes, as CADeSH does for benign traffic, to see whether recognizable benign types exist and whether those types recur across hospitals.
Feedback from Stage 2 to Stage 1. Prior pipelines run strictly forward. As an optional extension, we let the reviewer's uncertainty feed back into the tripwire. A patient whose recent triggers were confidently benign gets a temporarily raised $h$; an uncertain review lowers it. This extension is evaluated only after the base comparison.
Baselines
Naive reference. The do-nothing comparison is the tripwire alone, with every Stage 1 alarm counted as a sepsis alert. It shows how much the reviewer adds. Next to it we run a single-measurement, threshold-style rule on the same variables, which tests whether temporal tracking matters at all.
Strongest existing method. The strongest competitor is a challenge-style end-to-end model. We use gradient-boosted trees, plus a recurrent network, on all 40 PhysioNet variables with forward-filled values and missingness indicators. This family is representative of the leading entries in the PhysioNet 2019 challenge, whose top utility scores were 0.33–0.36 on the hidden test set that includes Hospital C.
That number is not directly comparable to our protocol. Hospital C is not public, so we evaluate leave-one-hospital-out between A and B, with MIMIC-IV as a third site using derived Sepsis-3 labels. The baseline is therefore retrained under our splits. The cited figure serves only as a sanity anchor: if our in-site end-to-end baseline falls far below it, we treat our reimplementation as suspect.
Fairness rules apply to every model. All systems see the same patients, the same horizon, and the same label definition. Each gets an equal hyperparameter search budget, counted in trials, tuned on training-site validation only. No model sees target-site labels.
Key ablation. The decisive control is the matched end-to-end model. It receives exactly the patient-relative features, the CUSUM statistics, the trigger indicator, and the class reweighting that the two-stage system uses, but it is trained on all hours as a single model. This control isolates the decomposition itself.
Two further ablations remove one ingredient each. The first replaces patient-relative normalization with population z-scores in both stages. The second gives the reviewer only the snapshot at trigger time, with no look-forward trajectory and no lab deltas.
What counts as winning. The two-stage system must show a smaller in-site-to-cross-site utility drop than the matched end-to-end control. It must also show equal or better cross-site utility. We will report confidence intervals from patient-level bootstrap and require that they exclude zero difference.
Why not just…?
Why not just train one end-to-end model with patient-normalized features and class reweighting? This is the real threat, and we take it seriously. Arguments from the security literature, that a single model trained on mostly normal data underweights rare anomaly classes (CSK-CNN; traffic detection), are partly answered by reweighting. They do not settle the question for clinical data. The matched control in our Baselines section is built to settle it. If the matched model transfers equally well, we will report that the benefit lies in patient-relative inputs and that the two stages buy only interpretability and fewer reviewer calls.
Why not just alarm on the tripwire, or on a static clinical cutoff? Benign transient spikes are common in exactly the variables a tripwire watches (D-dimer; white-coat BP), so we expect many alarms per patient-day. The naive baseline measures this directly, as alarm burden at matched sensitivity. Static cutoffs ignore trajectory direction, which mattered for albumin and hsCRP. The p-Tau217 result warns that a snapshot can sometimes suffice, so the snapshot-reviewer ablation tests whether trajectories help here specifically.
Why not use an LLM as the reviewer, as in metabolic pathway prompting? On Steroid-M, that approach reached 57.0% F1 with LLaMA 2-7B, against 40.3% for zero-shot prompting (MPP). Its results were sensitive to model scale, and it was not tested on irregular ICU records. A small trained reviewer is the cheaper first test. An LLM reviewer can replace it later on the same fired episodes, and the comparison will then be at a fixed trigger budget.
Why not run anomaly detection on TFT-multi forecast residuals directly? Forecast residuals are a reasonable tripwire signal, which is why we include them as a Stage 1 variant. Residuals alone, however, do not distinguish "resolved" from "persisted." TFT-multi is also reported to struggle with fast fluctuations in sparse SpO2 (TFT-multi). If residual-only alarms match the full system, the reviewer is unnecessary, and that experiment will show it.
Proposed Experiments and Decisive Checks
The primary experiment runs on PhysioNet 2019 Hospitals A and B in both train/test directions, and on MIMIC-IV as an external site. We report the challenge utility score, AUROC, and AUPRC at the patient-hour level. We also report alarms per patient-day at 80% episode sensitivity and median lead time before labeled onset.
Check 1: the recall ceiling. False negatives at Stage 1 cannot be recovered downstream. We therefore report the fraction of septic patients whose onset is preceded by at least one trigger within the horizon. We report it at the operating point used for the review, and separately for each site. If this ceiling falls well below the end-to-end model's sensitivity at the target site, the design fails regardless of reviewer quality.
Check 2: transfer. We compare the cross-site utility drop of the two-stage system against the matched end-to-end control.
Check 3: benign structure. We test whether resolved-episode clusters learned at one hospital still explain resolved episodes at the other, measured as the reviewer's false-positive rate on target-site resolved episodes.
Label caveat. Sepsis-3 onset times in retrospective records reflect documentation as much as biology (PhysioNet 2019). We therefore report lead-time results under small shifts of the onset label, to check that conclusions do not depend on exact timestamps.
Secondary check. If Steroid-M is obtainable, we will place the per-athlete tripwire in front of an LLM reviewer. We will count what fraction of profiles reach the LLM, and test whether F1 stays near the reported 57.0% while making fewer LLM calls.
References
- Early Prediction of Sepsis from Clinical Data: the PhysioNet/Computing in Cardiology Challenge 2019
- CADeSH: Collaborative Anomaly Detection for Smart Homes
- Association between albumin changes and prognosis in older patients with acute myocardial infarction
- Association Between High-Sensitivity C-Reactive Protein Trajectories and the Incidence of Metabolic Syndrome:A Retrospective Cohort Study
- The association between the neutrophil-to-lymphocyte ratio and mortality in critical illness: an observational cohort study
- Increased plasma D-dimer levels may be a promising indicator for diabetic peripheral neuropathy in type 2 diabetes
- Genetic associations vary across the spectrum of fasting serum insulin: results from the European IDEFICS/I.Family children’s cohort
- Physical activity and eating behaviors patterns associated with high blood pressure among Chinese children and adolescents
- Predicting Progression from Mild Cognitive Impairment to Alzheimer’s Disease using Survival Analysis
- Detect, Classify, Act: Categorizing Industrial Anomalies with Multi-Modal Large Language Models
- Temporal Opinion Spam Detection by Multivariate Indicative Signals
- Timeseries Anomaly Detection using Temporal Hierarchical One-Class Network
- Incorporating Metabolic Information into LLMs for Anomaly Detection in Clinical Time-Series
- TFT-multi: simultaneous forecasting of vital sign trajectories in the ICU
- CSK-CNN: Network Intrusion Detection Model Based on Two-Layer Convolution Neural Network for Handling Imbalanced Dataset
- An Abnormal Network Traffic Detection Algorithm Based on Big Data Analysis