Drift You Can Read: Discrete-State Dynamic Risk Models That Locate and Repair Their Own Temporal Decay

Hospital risk models get worse after deployment because medicine changes underneath them. Lab assays are swapped, treatments improve, and diagnostic definitions are rewritten. Today the standard response is to watch the area under the ROC curve (AUROC) fall and then retrain everything once enough new outcomes have piled up. We propose a dynamic risk model whose internal structure says where the decay comes from, and which can repair part of it before any new outcomes are observed.

The model routes each patient's risk through a small number of named physiological states, between five and eight. It has three separable pieces. The first is a description of what each state looks like in the data. The second is a table of how quickly patients move between states. The third is how dangerous each state is. These pieces can drift independently, and they need different repairs. The first two can be re-estimated from recent unlabeled patient timelines, while the third needs outcomes, which arrive late. Our central claim is that a structure like this turns "the model rotted" into an auditable statement. An example would be: "after 2014, patients leave the deteriorating state faster, while the deteriorating state is as lethal as before." We further claim that a label-free update of the first two pieces recovers a meaningful share of lost discrimination, which output recalibration provably cannot do.

Why this question, and why now

The temporal-shift literature has narrowed the space of plausible fixes, and three findings shape this proposal. First, a decade-long MIMIC-IV benchmark (Guo et al.) found that methods forcing features to look the same across time periods never beat plain training on pooled history. Those methods are invariance-based domain generalization and unsupervised domain adaptation. Their constraints threw away time-varying features that carried real signal. This finding rules out invariance as our mechanism: we never ask representations to stay fixed. Instead we let the generative parts of the model move while keeping the meaning of the states fixed.

Second, a systematic review (Guo et al., systematic review) found that recalibration fixes calibration decay but not discrimination decay. It also found that the only feature-level strategy that preserved discrimination without refitting was manual expert mapping of changing codes onto stable clinical concepts. We read this as the clearest motivating evidence. A stable, clinically named intermediate layer is what survives drift, and nobody has learned such a layer automatically for dynamic risk.

Third, EMDOT showed two things: random splits overstate performance, and complex models decay faster than logistic regression as data ages. This constrains us toward a low-capacity bottleneck and supplies our evaluation protocol.

On the modeling side, the closest work is the deep state-space framework for cancer EHR. It learns continuous latent states, clusters them afterward into dangerous, intermediate, and stable states, and fits a single fixed transition matrix over 2006–2018. It does not train states against outcomes, and it never asks whether the matrix changes with calendar time. Two other papers each supply half of what we want. DCSM assigns patients to a few interpretable parametric survival "experts," but only from baseline covariates. DySurv updates survival curves dynamically from ICU streams, but through an opaque continuous latent space. Neither was evaluated across deployment years. We combine DCSM-style named risk archetypes with Markov dynamics over ICU time series, and make calendar-era drift of each component the object of study.

Core Claim

Let $s_t \in \{1,\dots,K\}$ be a patient's hidden state at hour $t$. The model has three components. The first is a per-state observation model $p_e(x_t \mid s_t)$, which says what vitals and labs look like in each state during calendar era $e$. It factorizes over features so that each feature's per-state distribution can be plotted. The second is a transition matrix $A_e(s' \mid s)$, which can be shifted by a few covariates such as treatment indicators. The third is a per-state hazard $h_e(s)$ of the clinical event. Dynamic risk at landmark $t$ over horizon $\Delta$ is

$$P(T \le t+\Delta \mid T>t, x_{[0,t]}) = \sum_{s} \pi_t(s)\, F_{\Delta}(s; A_e, h_e).$$

Here $\pi_t$ is the filtered probability of each state given the history so far. $F_\Delta(s; A_e, h_e)$ is the probability of the event within $\Delta$ starting from state $s$, computed by rolling the chain forward. All training is end-to-end on the survival likelihood, so states are defined by risk rather than clustered afterward.

The claim has three parts. The first is attribution: given data from a new era, the change in each component can be estimated, and the resulting attribution recovers known injected drifts and aligns with documented real ones. The second is label-free repair: re-estimating $p_e(x\mid s)$ and $A_e$ by expectation-maximization on unlabeled recent timelines, with $h(s)$ frozen, closes a substantial fraction of the AUROC gap between a frozen model and a fully retrained one. The third is the anchor: freezing hazards is what keeps "state 3" meaning the same thing across eras. Without that anchor, unsupervised re-estimation relabels the states and the audit becomes meaningless.

Research Program

We will build the model with observation densities that factorize over features. Our default is per-feature piecewise or mixture densities with explicit missingness indicators, so that hospital monitoring intensity is modeled rather than ignored. The transition matrix is a small softmax-parameterized table. We will fit it with a forward-filtering likelihood that handles irregular sampling, using hourly cycles and censoring at discharge.

Interpretability lives in three artifacts that clinicians can inspect directly. For each state, these are the per-feature distributions and the hazard. Across states, the artifact is the transition table. For each era, it is a drift report built from two quantities: the divergence of each feature's per-state distribution from the reference era, and the change in each transition probability. Each is reported with bootstrap intervals.

The analysis component is a decomposition of observed performance change between eras into three counterfactual contributions. We swap in one re-estimated component at a time while holding the others at reference values. We also report the interaction remainder rather than hiding it.

The repair component compares four update policies against the same temporal splits: no update, label-free update of observation model and transitions, hazard-only update using whatever labels have arrived, and full retraining. This gives practitioners a concrete schedule: they can update the observation model and transitions continuously, and update hazards only when outcomes mature.

Baselines

The naive reference point is a frozen model. Every method is trained on MIMIC-IV's 2008–2010 window and evaluated unchanged on 2011–2013, 2014–2016, and 2017–2019. This follows the era structure of Guo et al., who report a maximum AUROC drop of 0.090 for sepsis under pooled training. That paper used static admission-window prediction, so their 0.090 is a scale reference rather than a directly comparable number for our hourly dynamic task. We will reproduce the static setup first to confirm we see a drop of similar size before moving to the dynamic task.

The strongest maintenance competitors are pooled-history training and sliding-window training, as defined in EMDOT. We will instantiate both with a GRU with a Weibull survival head and with DySurv (Mesinovic et al.). These are the right competitors because Guo et al. found nothing that beats training plus periodic retraining. DySurv is reported to beat Cox-Time and Dynamic-DeepHit on MIMIC-IV, but no numerical AUROC or C-index for it is given in the file. On eICU it reports time-dependent concordance "over 60%." No temporally split dynamic MIMIC-IV result was found on Lacuna for any of these methods, so we will produce those numbers ourselves.

The strongest simple dynamic baseline is Last Visit features with a Random Survival Forest under super landmarking, as in de Swart et al.. That paper reports a time-dependent AUC of 0.96 and a Brier score of 0.07 on ADNI, which we will reproduce as a sanity check of our pipeline. We will also run a landmarked logistic regression, given EMDOT's finding that simple models age better.

The strongest interpretable competitor is the survival-stacked Explainable Boosting Machine of Van Ness et al.. An Explainable Boosting Machine is an additive model of per-feature curves; that paper reports a time-dependent AUC of 0.834 against 0.815 for Cox on heart failure. We will apply it to Last Visit features at each landmark.

Every method gets the same features, the same validation data drawn only from the training era, and the same hyperparameter search budget (an equal number of configurations). The same labels are available at each update point, and update policies are matched on label access.

The model counts as successful if three things hold. First, its in-era time-dependent AUC must be within 0.02 of the best black-box baseline, a tolerance we set in advance as the acceptable price of the bottleneck. Second, its label-free update must recover at least half of the frozen-to-retrained AUROC gap in late eras, where recalibration recovers essentially none by construction. Third, its fully retrained version must not fall below sliding-window GRU performance. The key ablations are an unanchored update, in which hazards are re-estimated jointly without labels via EM, and a model with no transition structure. The second scores each hour's state independently, which isolates whether the Markov dynamics add anything beyond a state-based Last Visit model.

Why not just…?

Why not just retrain periodically on recent data? This is the right default, and it remains our upper reference. Its weaknesses are that it needs matured outcomes, so it lags drift by the outcome horizon, and that it says nothing about what changed. If label-free repair recovers little of the gap, our practical claim fails and only the auditing claim remains. The policy comparison above settles this directly.

Why not just use Last Visit plus a Random Survival Forest? De Swart et al. show that the current snapshot often dominates near-term risk, and this is a real threat to the Markov component. Our model's filtered state is itself a compressed "current state," so we expect discrimination to be comparable, not better. The question is whether transitions buy repair and audit value. The no-transition ablation answers exactly this. If it matches the full model on both drift recovery and attribution accuracy, we will drop the Markov dynamics and report a state-bottleneck Last Visit model instead.

Why not just recalibrate? The systematic review shows that recalibration restores calibration but not ranking when the relationship between features and outcomes moves. We will include Platt and isotonic recalibration as a baseline for calibration error. We will also apply recalibration on top of our own updates, since the two are complementary rather than competing.

Why not just fit an Explainable Boosting Machine per era and compare the shape curves? This is the most dangerous alternative, because it also yields a readable drift audit at low cost. Its weakness is confounding. A changed age curve could reflect a different case mix, faster recovery, or a changed definition of the outcome, and the curves cannot separate these. It also needs outcome labels in the new era. We will test it head-to-head on the synthetic attribution task described below. If the per-era curve differences localize the injected drifts as well as our component-wise decomposition does, that is a negative result we will report.

Proposed Experiments

The decisive check is semi-synthetic. On MIMIC-IV 2008–2010 patients, we inject three known drifts one at a time. Observation drift is a unit or offset change to a lab such as lactate. Transition drift is a uniform speedup of exits from high-risk states, implemented by resampling trajectories under a fitted model. Hazard drift is a relabeling of outcomes mimicking a definition change. We then measure whether the decomposition assigns most of each performance change to the correct component, and whether the right repair policy fixes it. Label-free repair should fix the first two drifts and should fail on the third. That failure is a prediction that would falsify the anchoring logic if it did not occur.

The real-data experiment runs dynamic hourly in-hospital mortality and sepsis-onset prediction across the four MIMIC-IV eras. We report time-dependent AUC, Brier score, and expected calibration error per era and per update policy, plus the fraction of the frozen-to-retrained gap closed by each policy. For sepsis, the transition to Sepsis-3 that Guo et al. document is an external marker. Our audit should attribute late-era sepsis decay mainly to the hazard and label component rather than the observation model, and we state this prediction in advance.

The cross-check is ADNI, a slower-moving disease cohort. On ADNI we compare against the 0.96 time-dependent AUC of the Random Survival Forest baseline, in a setting where states such as cognitively normal, mild impairment, and dementia have clinical names we can test our learned states against.

Clinician review closes the loop. Two intensivists will be shown each era's drift report blinded to the era and asked to order the reports chronologically and identify the main change. This gives a cheap, falsifiable test of whether the audit is legible, as opposed to merely computable.