A clinical alert rule, such as "page the team when lactate exceeds a cutoff" or "flag sepsis when the model's risk score crosses 0.5," is normally judged by how well it separates sick from healthy patients in one cohort, for example by the Youden index (sensitivity plus specificity minus one). That number does not say how fragile the rule is. Measured values reach the rule through assays that differ across platforms and drift over time. Patients also traverse moments where the decision hinges on a hair. We propose measuring a different property of any biomarker threshold rule, its sequential flip budget. This is the smallest amount of plausible, time-correlated measurement distortion that changes when, or whether, the alert fires. We compute it with the same optimal-control machinery that adversarial machine learning uses to steer forecasting models. We then ask two questions. Do thresholds chosen to maximize this budget transfer across hospitals better than thresholds chosen by the Youden index? And do the trajectory-based rules that clinicians increasingly favor turn out to be cheaper to flip than static cutoffs?
The idea joins two literatures that currently do not talk to each other. Adversarial work on autoregressive models shows that small perturbations, coordinated over time, can steer forecasts to arbitrary targets far more cheaply than greedy or random noise. Biomarker standardization work shows that cutoffs drawn from one cohort fail elsewhere, partly because of assay heterogeneity and baseline drift. We treat the adversary as a worst-case model of that heterogeneity rather than as a hacker. The resulting audit is cheap enough to run on any threshold rule before it is deployed.
Core Claim
Many clinical alert rules are linear filters of recent measurements followed by a cutoff. Static cutoffs, exponentially weighted moving averages, slope rules and autoregressive forecasts all fit this description. For such rules, the minimum distortion that suppresses or triggers an alert is an exact convex program, a direct clinical analogue of the poisoning formulation of Data Poisoning Attacks against Autoregressive Models. We claim three things. First, the flip budget varies by large factors across rules with similar AUROC, and Youden-optimal cutoffs systematically sit at low-budget points. Second, masking an alert (a false negative) is cheaper than inducing one, mirroring the spike/dip asymmetry Alfeld et al. found. Third, choosing the cutoff to maximize clinical utility under the worst-case distortion yields thresholds whose performance degrades less on an unseen hospital or time period, and this holds without using any labels from the new site.
The Measurement
Let $x_{1:T}$ be a patient's hourly biomarker series. Let $\delta_{1:T}$ be a distortion added to it. The rule raises an alert at hour $t$ when $s_t = w^\top (x+\delta)_{t-k:t} > \tau$. Here $w$ is the rule's filter weights over the last $k$ hours, and $\tau$ is the cutoff. We restrict the distortion to what an assay or calibration error could plausibly produce. Each step is bounded, $|\delta_t| \le \epsilon$, and the distortion changes slowly, $|\delta_t - \delta_{t-1}| \le \rho$. Both bounds are set from empirical within-patient repeat variability and from the cross-hospital distribution gaps that Early Prediction of Sepsis from Clinical Data: the PhysioNet/Computing in Cardiology Challenge 2019 documents in variables such as EtCO2 and TroponinI. For a septic patient whose alert would fire inside the clinically useful window $W$ before onset, the masking budget is
$$ B_{\text{mask}} = \min_{\delta} \|\delta\|_2 \quad \text{s.t.} \quad w^\top (x+\delta)_{t-k:t} \le \tau \;\; \forall t \in W, \;\; |\delta_t|\le\epsilon, \;\; |\delta_t-\delta_{t-1}|\le\rho. $$
All constraints are linear in $\delta$, so this is a small quadratic program that can be solved exactly per patient. The false-alarm budget $B_{\text{alarm}}$ flips the inequality for one hour of a non-septic stay. When no feasible $\delta$ exists inside the box, the rule is certified robust for that patient. For nonlinear models such as gradient-boosted trees or recurrent networks over all 40 PhysioNet variables, exact solutions are unavailable. In that case we compute upper bounds on the budget with projected gradient descent over the same constraint set. For trees we use a differentiable distilled surrogate, the route that Adversarial Attacks on Time Series used to attack nearest-neighbor classifiers.
Prior Work and How This Differs
Three adversarial papers supply the method. Data Poisoning Attacks against Autoregressive Models casts poisoning of an AR model as a quadratic program and finds that inducing spikes is cheaper than inducing dips. Optimal Attack against Autoregressive Models by Manipulating the Environment uses linear-quadratic control to steer forecasts, reporting a mean cost of 11.5 against 1492 for a greedy attacker on an oscillating AR(3) system. Adversarial Vulnerabilities in Large Language Models for Time Series Forecasting shows directional perturbations within 2% of the data mean raise LLMTime (GPT-4) MSE on IstanbulTraffic from 0.805 to 1.417, while random noise of the same size reaches only 0.883. All three study forecast error on economic or infrastructure series. We instead target a binary decision boundary on clinical biomarkers, under a constraint set tied to measurement physics. We also use the attack as an instrument for choosing thresholds, not as a demonstration of harm.
Strategic-timing work tells us where to expect fragility. Kos et al. disable RL policies by perturbing a few frames. Xu et al. locate "bottleneck states" where small margins decide long rollouts. We test whether the hours just before sepsis onset are such bottlenecks, which would predict that the optimal masking distortion concentrates there.
The closest clinical attack is Longitudinal Mammogram Exam-Based Breast Cancer Diagnosis Models: Vulnerability to Adversarial Attacks (see also; Longitudinal Mammogram Exam-Based Breast Cancer Diagnosis Models: Vulnerability to Adversarial Attacks; Longitudinal Mammogram Exam-Based Breast Cancer Diagnosis Models: Vulnerability to Adversarial Attacks). It drives a two-exam model's AUC to 0.205, and to 0.577 under adversarial training. It works on image pairs with a fixed decision threshold and never asks how the cutoff itself should be chosen. That last question is the one we answer.
On the standardization side, the papers motivate the problem rather than supply methods. Feng et al. set an NLR cutoff at the cohort median of 3.24 and note it may not generalize. Wei et al. find AGR cutoffs ranging from 1.14 to 1.93 across 12 cohorts with $I^2 = 76.8\%$. Wan et al. and Misra et al. attribute such spread partly to assay platforms. Finding Optimal Cutoffs for Continuous Biomarkers and related work formalize the Youden approach we compare against. Tracking Dynamic Biomarker Trajectories Over Time, Zhang et al. and Pan et al. argue that trajectory direction beats snapshots.
The trajectory literature carries a hidden tension that our measurement exposes. A slope rule differences successive values, so a slow ramp distortion of size $\rho$ per hour shifts its score directly. A static cutoff needs the full level shift. Trajectory rules may therefore be more predictive and also cheaper to flip.
One paper constrains the design. Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine shows that invariance-based training fails to beat plain risk minimization under natural drift on MIMIC-IV, with sepsis AUROC dropping by up to 0.090. Because of this, we do not propose another invariant-representation learner. We keep the model fixed and change only the cutoff and the rule family, which is where standardization decisions are actually made.
Research Program
The work has three parts. The first is an audit. For each rule family and each cohort, we compute per-patient masking and false-alarm budgets with the exact program for linear rules and gradient upper bounds for learned models. We report the distribution of budgets, the fraction of patients certified inside clinically calibrated bounds, and where in time the optimal distortion concentrates. The second part is robust threshold selection. We choose $\tau$, and optionally the filter $w$ from a small family, to maximize the PhysioNet utility evaluated on the worst-case distorted training data rather than on the clean data. This is an outer one-dimensional search over cutoffs, with the inner program solved per patient. The third part is a transfer test. Thresholds fit on PhysioNet hospitals A and B are evaluated on hidden hospital C. Thresholds fit on MIMIC-IV 2008–2010 are evaluated on 2017–2019, using the MIMIC-Extract preprocessing so units and outlier clipping are standardized. The question is whether robust thresholds lose less utility and less AUROC than Youden thresholds, without any target-site labels.
Baselines
The naive reference is a fixed cutoff chosen by the Youden index on the training cohort, as in Finding Optimal Cutoffs for Continuous Biomarkers, alongside a cohort-median cutoff in the style of Feng et al. These are what most clinical studies deploy. Beating them means higher utility on hospital C and on the 2017–2019 MIMIC-IV epoch at matched alert rate.
The strongest existing predictive systems are the PhysioNet 2019 challenge models. For them, Reyna et al. report winning normalized utility scores between 0.33 and 0.36 on the hidden test set. We reimplement a gradient-boosted model of that type on the official A/B training split with the same feature set and tuning budget. We then compare its native thresholding against robust threshold selection applied to the same scores. Matching the 0.33–0.36 range is the sanity check. The claim requires the robust cutoff to retain at least that utility on clean C data while losing less under distortion.
For the audit itself, the key ablation isolates the temporal structure of the adversary. We compare the optimal sequential distortion against three weaker distortions under identical $\epsilon$ and $\rho$ bounds: a constant offset (the pure calibration-bias model), random correlated noise, and a greedy hour-by-hour perturbation. If the sequential optimum does not find budgets substantially smaller than these, the optimal-control machinery adds nothing. In that case the flip budget reduces to ordinary sensitivity analysis.
For the selection procedure, the ablation trains robust thresholds against the constant-offset set only. This tells us whether the time-varying part of the distortion set matters for cross-site transfer.
Why not just…?
Why not just recalibrate the cutoff at each new hospital? With labeled outcomes from the new site, local recalibration is the gold standard and will likely win. We include it as an oracle upper bound. Our procedure targets the common case where a rule must be set before local labels exist, and success means closing a meaningful share of the gap between Youden thresholds and this oracle.
Why not just match distributions across sites, for example quantile-normalizing each biomarker at the new hospital? This is a real threat. Reyna et al. show that cross-hospital differences appear as CDF shifts, and label-free quantile matching removes constant and monotone distortions directly. We run it as a baseline on hospital C and on MIMIC-IV epochs. If it equals robust thresholds on transfer, we drop the transfer claim. We would retain only the audit, which quantile matching cannot provide, because within-stay drift and timing-concentrated distortions are not visible in marginal distributions.
Why not just stress-test with random noise or a fixed offset? That is the ablation above. The forecasting evidence of Liu et al., where directional perturbations raise MSE far more than equal-size noise, and of Chen et al., where optimal control costs two orders of magnitude less than greedy attack, predicts that random testing badly overestimates robustness. Our audit measures whether that gap holds for clinical alert rules.
Why not just smooth the inputs or adversarially train the model? Smoothing is itself a linear filter, so it falls inside our rule family and gets audited directly. A slow ramp passes through any low-pass filter. Adversarial training of the full model is the defense that still left an AUC of 0.577 in Ao et al. It also changes the model rather than the cutoff, and Guo et al. caution that robustness-oriented training on clinical drift has not beaten plain training. We include it as an additional arm on the recurrent model only.
Decisive Checks
The project succeeds if three things hold. Flip budgets differ by at least several-fold among rules with AUROC within a few hundredths of each other. Masking budgets fall below false-alarm budgets in a majority of rule–cohort pairs. Robust thresholds lose significantly less utility than Youden thresholds when moving from PhysioNet A/B to C and from MIMIC-IV 2008–2010 to 2017–2019, and also beat quantile matching. Significance is judged by paired patient-level bootstrap.
A specific test of the trajectory tension asks whether slope-based rules, despite better clean AUROC, show lower median masking budgets than static cutoffs on lactate and creatinine. If they do, standardization guidance favoring trajectories needs a robustness caveat. If the transfer result fails while the audit results hold, the contribution narrows to a pre-deployment fragility measure for biomarker thresholds, which is still more than any cited standardization study currently reports.
References
- Data Poisoning Attacks against Autoregressive Models
- Early Prediction of Sepsis from Clinical Data: the PhysioNet/Computing in Cardiology Challenge 2019
- Adversarial Attacks on Time Series
- Optimal Attack against Autoregressive Models by Manipulating the Environment
- Adversarial Vulnerabilities in Large Language Models for Time Series Forecasting
- Delving into adversarial attacks on deep policies
- COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
- Longitudinal Mammogram Exam-Based Breast Cancer Diagnosis Models: Vulnerability to Adversarial Attacks
- Longitudinal Mammogram Exam-Based Breast Cancer Diagnosis Models: Vulnerability to Adversarial Attacks
- Preoperative Neutrophil-to-Lymphocyte Ratio as a Predictive and Prognostic Factor for High-Grade Serous Ovarian Cancer
- Low Pretreatment Albumin-to-Globulin Ratio Predicts Poor Prognosis in Gastric Cancer: Insight From a Meta-Analysis
- Plasma fibrin D-dimer and the risk of left atrial thrombus: A systematic review and meta-analysis
- Prognostic biomarkers of intracerebral hemorrhage identified using targeted proteomics and machine learning algorithms
- Plasma 1,5-anhydro-d-glucitol is associated with peripheral nerve function and diabetic peripheral neuropathy in patients with type 2 diabetes and mild-to-moderate hyperglycemia
- The association between the neutrophil-to-lymphocyte ratio and mortality in critical illness: an observational cohort study
- Association between albumin changes and prognosis in older patients with acute myocardial infarction
- Association Between High-Sensitivity C-Reactive Protein Trajectories and the Incidence of Metabolic Syndrome:A Retrospective Cohort Study
- Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine
- MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III