Does Treatment Synergy Drift, or Only the Floor Beneath It?

When two treatments are given together, a hospital usually wants to know whether the pair does more than either one alone. A retrospective cohort study answers this from historical records: it looks back at patients who did or did not receive each component and compares their outcomes. Those records span years, and over that time sicker and healthier patients arrive, supportive care improves, and billing codes change. A combined treatment that cut mortality by ten percentage points in 2009 may cut it by four in 2019. It is unclear whether the pair stopped working together or whether everyone's baseline risk simply fell, leaving less room to improve. In aggregate COVID-19 data, Sharma et al. found the second pattern: the same interventions removed less transmission once earlier precautions had lowered the "safety floor." Nobody has asked the same question at the patient level, in clinical bundles, using the routine administrative records hospitals actually hold.

We propose to test a specific, falsifiable structural claim on more than a decade of intensive care records. The claim is that, on the right scale, the multiplicative interaction between bundle components is roughly stable across calendar eras, and that most of the apparent drift in absolute synergy comes from a drifting baseline risk. If the claim holds, current-era bundle effects can be forecast from old-era interaction estimates plus a cheap, one-parameter update of baseline risk fitted on a few hundred new patients. That would be far earlier and less noisy than re-running a full causal study in each new era. If it fails, we will have measured how fast true synergy itself decays, which retrospective cohorts that pool decades of data currently ignore.

Core Claim

Let $r_e(a, x)$ be the risk of in-hospital death for a patient with baseline record $x$ in calendar era $e$, had they received bundle configuration $a \in \{00, 10, 01, 11\}$ (neither component, one, the other, both). The hypothesis is that

$$r_e(a, x) = r_e(00, x)\,\rho(a, x),$$

where $\rho(a, x)$ is an era-invariant ratio, and that $r_e(00,x)$ shifts across eras mainly through a single era-specific intercept. Under this structure the absolute interaction contrast, $r_e(00,x)\,[\rho(11,x) - \rho(10,x) - \rho(01,x) + 1]$, shrinks in proportion to the baseline. This is the patient-level analogue of the safety-floor effect, and of the multiplicative composition $R_t = R_0\gamma_1\gamma_2$ used by Tian et al. for transmission. Which scale is invariant (risk ratio, odds ratio, or neither) is an empirical question that we settle on held-out eras rather than assume.

The claim connects to a known result in prediction. The systematic review by Guo et al. found that temporal shift damages calibration (whether predicted probabilities match observed rates) before it damages ranking, and that simple recalibration repairs calibration but not ranking. Our hypothesis is the causal counterpart. If bundle drift is mostly "calibration-in-the-large" of the baseline, a one-parameter update should transport bundle effects. If drift lives in $\rho$, which is the causal analogue of a change in the feature–outcome relationship $P(Y|X)$, no cheap update will suffice.

Positioning Against Prior Work

The closest work falls into three groups. Sharma et al. motivate the question. They fit a hierarchical Bayesian renewal model to 114 European regions and found that the combined effect of 17 interventions fell from 77–82% to 66% between waves. That model works on regional case counts, not patient records, and it does not separate a falling floor from a changing interaction.

Their data also partly contradicts our hypothesis. Effects shrank even on the multiplicative $R_t$ scale, and school closures fell to about 7%. Multiplicative invariance is therefore not guaranteed, and our design treats its failure as a reportable outcome.

The second group supplies methods but assumes stationary data. The Medical Deconfounder fits factor models to medication logs to build stand-in variables for unmeasured patient severity, and its authors list the lack of temporal modelling as a limitation. Variational Sample Re-weighting handles high-dimensional bundles through a learned latent space, but it was tested only on synthetic and recommender data with identical train and test distributions. The worst-case subpopulation treatment effect gives an efficient estimator that is robust to covariate shift, but only for a single binary treatment. We borrow from these estimators but study a different object: how bundle effects move across calendar time.

The third group constrains the design. Guo et al. 2022 showed on MIMIC-IV that invariance-seeking training did not beat pooled ordinary training for prediction, so we deliberately avoid invariance penalties. Kaushik et al. learned fluid–vasopressor dosing with conservative offline reinforcement learning on MIMIC-III. The shift they handle is the gap between the learned dosing policy and physicians' behaviour, not drift across calendar years, and they report no formal effect estimates. Their bundle is our natural test case.

Research Program

The cohort is adult sepsis-3 admissions in MIMIC-IV, split into the four triennia used by Guo et al. 2022: 2008–10, 2011–13, 2014–16 and 2017–19. The bundle has two components measured in the first 24 hours: any vasopressor, and fluid volume above the cohort median. The outcome is in-hospital mortality.

Confounders come from administrative and routine data: ICD diagnosis codes, prior procedures, the first-day laboratory panel, and the first-day severity score. This tests the claim that claims-style data carry usable frailty signal (Rana et al.; Ahmad et al.). Because MIMIC-IV crosses the ICD-9 to ICD-10 transition, we map codes to clinician-defined phenotype groups. The review by Guo et al. found this kind of expert grouping was the only feature-level fix that preserved model ranking across time; automated compression such as PCA did not.

Within each era we estimate the four arm-specific risks with augmented inverse-probability weighting (AIPW), a doubly robust method that combines an outcome model with reweighting by treatment probabilities. Both nuisance models are gradient-boosted, fitted with cross-fitting, and used with overlap trimming. A sensitivity variant adds Medical Deconfounder substitute confounders built from medication logs.

The structural model is then fitted on the three earliest eras. It has a shared ratio $\rho(a,x)$, parameterized either as a log risk ratio or a log odds ratio with interactions in baseline risk deciles, plus a free intercept per era. We test era-invariance of $\rho$ with a likelihood-ratio test and by checking whether ratios estimated from each pair of eras predict the remaining early era.

The transport step uses the 2017–19 era. We take the fitted $\rho$, estimate only the new era's intercept from $n$ new patients, and forecast all four arm risks and the interaction contrast for that era. We vary $n$ from 100 to the full era.

Finally, a semi-synthetic study keeps the real MIMIC-IV covariates and treatments but simulates outcomes. In one condition the floor drifts and $\rho$ is fixed; in another $\rho$ also drifts at controlled rates. This gives known ground truth and shows when the method breaks.

Baselines

The naive reference is the pooled static estimate. It is an AIPW estimate of the bundle effects fitted on all of 2008–16 and applied unchanged to 2017–19. It mirrors the single-propensity-score practice of long retrospective cohorts such as Tewari et al., and the pooled ordinary training that won in Guo et al. 2022.

The strongest competitor is full re-estimation. This means running the same AIPW estimator from scratch on $n$ patients from the new era. It is the honest gold standard once enough data exist, and its weakness is variance at small $n$.

A third baseline adds calendar time to the pooled AIPW outcome model, as a linear trend that also interacts with each bundle arm, and extrapolates to 2017–19. This lets every quantity drift smoothly.

All three baselines and the proposal share the same confounders, cross-fitting folds, gradient-boosting tuning budget and trimming rule. Each method's forecast for 2017–19 is scored against the full-era AIPW estimate, with that estimate's own sampling variance subtracted, and against known truth in the semi-synthetic runs.

The key ablation is "intercept-only drift with era-specific $\rho$." It isolates whether a shared interaction is what drives any gain. A second ablation swaps risk-ratio and odds-ratio parameterizations to identify which scale, if either, is invariant.

The proposal beats the field if, at $n \le 500$ new patients, its error on the interaction contrast is lower than full re-estimation's with non-overlapping bootstrap intervals, and it is never worse than pooling. It must also beat the time-trend model on held-out error.

Numbers from the file give context only. For prediction on this cohort, Guo et al. 2022 report a maximum sepsis AUROC drop of 0.090 (95% CI 0.080–0.101) from 2008–10 to 2017–19, and an AUROC difference of −0.003 to 0.050 for invariance methods relative to pooled training. There is no reported result found on Lacuna for transporting or forecasting bundle treatment effects across eras on MIMIC-IV.

Why not just…?

Why not just pool all years and estimate once? Pooling is the right choice if neither floor nor ratio drifts, and the pooled baseline tests that directly. The prediction literature shows pooled models still lose ranking over time (Guo et al. 2022). Sharma et al. show that intervention effects measured in different eras differ materially, so a pooled average describes neither era. If pooling matches our forecast error, the honest conclusion is that this bundle did not drift.

Why not just re-run the causal analysis in each new era? This is the real threat, and with a full era of data it should win or tie. Our claim concerns small $n$: a one-parameter update has far lower variance than four arm risks plus an interaction re-estimated from scratch. The learning curve over $n$ will show the crossover point. If full re-estimation is already competitive at a few hundred patients, the practical case for the proposal largely disappears and we will say so.

Why not just add calendar year and its interactions with treatment to the regression? A flexible trend can absorb drift in both floor and ratio, so it is a legitimate competitor. It has more parameters and extrapolates trends rather than structure, which makes it fragile at a step change such as the ICD transition or a guideline revision. The held-out-era comparison and the semi-synthetic step-change condition decide between the two. If the trend model wins, drift is smooth enough that structure adds nothing.

Why not just fit a product interaction term in logistic regression? This is exactly one of our candidate scales, the odds-ratio version of $\rho$. On its own, it does not adjust well for confounding by indication; Zhang et al. report that 12 of 20 naive associations were spurious. It is also not tested for stability across eras. We keep the scale choice as a hypothesis and embed it inside the doubly robust estimator.

Decisive Checks

Three results would settle the claim. The first is the invariance test on the early eras. If the ratios differ significantly across 2008–16 on both scales, the hypothesis fails before transport is attempted, and we report the magnitude of synergy drift as the main finding.

The second is the 2017–19 forecast. This checks whether the shared-ratio, intercept-only model forecasts the held-out interaction contrast within the full-era estimate's confidence interval, and beats full re-estimation below roughly 500 patients.

The third is a negative control on the semi-synthetic data. When $\rho$ is made to drift, the method's error must grow as predicted, and the invariance test must flag it. Without this, a forecast that looks good on real data could be explained by luck.

A secondary replication on the European NPI registry of Sharma et al. will ask the same floor-versus-ratio question at the population level. That data already suggests partial failure, which makes it a useful stress test rather than a confirmation.