Forgiving Control: Noise-Robust Resuscitation Policies for Traumatic Brain Injury

When a patient with severe traumatic brain injury (TBI) reaches the intensive care unit, the deadliest events are often not the initial impact but the "second hits" that follow: transient blood-pressure drops that restart a cascade of neuronal death hours or days after admission. Observational data from 65 European centers show that patients who experience even one such systemic insult, an episode of systolic blood pressure below 90 mmHg, have 31.8% mortality versus 19% for those who do not, along with sharply elevated markers of neuronal death (Early systemic insults following traumatic brain injury). Yet nobody has built a decision system that manages this problem as what it actually is: a noisy, sequential control problem, where the drug dose a clinician orders is not reliably the dose the patient receives, and where today's actions shift tomorrow's injury biology.

This proposal builds such a system. The core idea is to take three lessons that live in separate literatures and combine them in one offline reinforcement learning pipeline for TBI hemodynamic management, evaluated across geographic registries. From the repeated-games literature on the "trembling hand," we take the finding that policies which cannot absorb execution errors collapse into destructive spirals, and that explicit recovery structure, calibrated forgiveness, and distinguishing slips from intent are what prevent this (Achieving Sustainable Cooperation in Generalized Prisoner's Dilemma with Observation Errors; When apology is sincere, cooperation evolves, even when mistakes occur frequently). From the receptor-pharmacology literature, we take the principle that a single drug is really several interventions bundled together, and that therapeutic effect can be decoupled from on-target toxicity only if the action representation separates them (The Role of Glutamate Receptors in Ischemic Stroke; A novel neuroprotective derived peptide of erythropoietin improved cognitive function in vascular dementia mice). And from the geographic-generalizability literature, we take the constraint that anything learned in one region's trauma system must survive transfer to another (Mortality Prediction in Severe Traumatic Brain Injury).

Core Claim

The central claim has two parts, one methodological and one empirical. Methodologically: offline reinforcement learning for critical-care drug titration should be trained in an environment model that explicitly injects execution noise — delayed, dropped, or attenuated titration steps — and should represent drug actions in a decoupled form that separates the therapeutic pathway from the toxicity pathway, mirroring how selective receptor antagonism separates beneficial from harmful signaling. Empirically, we predict a specific failure mode in the strongest existing approach: Conservative Q-Learning, the value-conservative algorithm behind the best current vasopressor dosing policies (Realistic CDSS Drug Dosing with End-to-end Recurrent Q-learning for Dual Vasopressor Control), penalizes uncertain returns. Game theory has shown that exactly this kind of return-level risk aversion paradoxically shrinks the basin of cooperative behavior under noise, while penalizing the variance of policy updates instead expands it (The Price of Paranoia). We hypothesize an analogous "conservatism paradox" in care: a value-conservative agent, seeing that aggressive hemodynamic stabilization produces high-variance outcomes (because some titrations slip), will learn to under-treat precisely the patients most vulnerable to second hits. Noise-aware training with recovery structure should fix this, and the fix should be measurable as graceful rather than catastrophic value degradation as the injected error rate rises.

Formally, if the clinician-agent orders dose adjustment $a_t$ at step $t$, we train against an execution model where the implemented action $\tilde{a}_t$ differs from $a_t$ with probability $\epsilon$: it is delayed one or more steps, dropped, or attenuated. The learning objective is the standard discounted return under this noisy transition, with the action space factored so that a hemodynamic agent and a neuroprotective agent each control their own pathway-specific channel rather than a monolithic dose vector.

Research Program

Three components. First, a noise simulator layered on real ICU trajectories. Using the CENTER-TBI core database, whose 1,695 adult ICU patients across 65 European centers carry hourly physiology, therapy-intensity records, and first-48-hour biomarker panels (GFAP, S100B, UCH-L1, NSE, NfL, Tau) plus 6-month Glasgow Outcome Scale-Extended scores (Early systemic insults following traumatic brain injury), we construct an environment model in which titration commands are corrupted at controlled rates $\epsilon$, spanning the 1%–40% range examined in the game-theory literature (When apology is sincere; The Price of Paranoia). We must be candid about a data limitation: if per-hour vasopressor titration timestamps in CENTER-TBI prove too sparse, we will develop and validate the method on the eICU and MIMIC-IV databases, the same pair used for dual-vasopressor control in sepsis (Realistic CDSS Drug Dosing), and evaluate the TBI-specific injury-cascade reward on CENTER-TBI. Second, a selectivity-decoupled action architecture. Where the prior dosing work treats vasopressors as blunt stabilizers, we factor the action into a perfusion-restoration channel and a neuroprotection channel whose toxicity penalties are grounded in the pharmacology: full-length erythropoietin raised red blood cell counts by 30% in the BCAS mouse model while the receptor-selective mimetic DEPO left hemoglobin completely unchanged over eight weeks (A novel neuroprotective derived peptide), and non-selective TNF blockade destroys repair signaling that a TNFR1-selective domain antibody preserves (Novel anti-TNFR1 domain antibody). The reward therefore penalizes monolithic high-dose actions more than pathway-targeted ones, and the policy must learn that the cheap-looking option carries a hidden cost. Third, policy evaluation and transfer. All policy comparisons use doubly robust off-policy estimation, because the trauma-registry literature shows standard inverse propensity weighting fails catastrophically under confounding by indication: on the tranexamic acid question, IPW estimated a 9–10% mortality increase where the doubly robust estimator recovered −0.01, matching the CRASH-3 randomized trial (Machine Learning Augmented Causal Inference). Transfer is tested by training on the CENTER-TBI China Registry (2,804 severe TBI patients) and evaluating on the European registry (1,113 patients), the exact direction in which XGBoost mortality prediction was shown to transfer with a C-statistic of 0.88 (Mortality Prediction in Severe Traumatic Brain Injury).

Baselines

The naive reference point is recorded clinical practice itself: the behavior policy embedded in the registries, together with the static guideline of maintaining systolic pressure at or above 90 mmHg. This is not a strawman — it is what patients actually receive, and the systemic-insult analysis shows it is too permissive, letting second hits through that carry a 31.8% versus 19% mortality gap (Early systemic insults). Beating this baseline means a doubly robust estimated 6-month outcome distribution that is strictly better than the behavior policy's, with bootstrap confidence intervals excluding zero.

The strongest existing competitor is the recurrent Conservative Q-Learning architecture with stepwise-directional action spaces developed for dual vasopressor control in septic shock (Realistic CDSS Drug Dosing), which reports more than a 3x improvement in weighted-importance-sampling reward over continuous-dosing baselines on eICU and MIMIC-IV (no exact per-table WIS values are reported on Lacuna for that paper). We will re-implement it faithfully, retrain it on the same TBI data, features, compute budget, and hyperparameter search as our method, and treat it as beaten only if our noise-trained, decoupled policy achieves higher doubly robust estimated value in the noise-free evaluation and degrades substantially less at injected error rates of 20% and 40% — the regime where risk-sensitive variants in the game-theory setting collapsed to cooperation around 0.5 while update-variance-targeted training held above 0.9 (The Price of Paranoia).

Two ablations isolate our ingredients. Training the identical architecture with a deterministic execution model (no noise injection) tests whether noise-awareness is doing the work or merely adding complexity. Training with a monolithic dose vector instead of the selectivity-decoupled channels tests whether the pharmacological factoring matters. A third variant, applying Entropic Value-at-Risk to returns in the style the game-theory work showed to be harmful, is included deliberately as a predicted failure, testing the conservatism-paradox hypothesis rather than our method.

Why not just…?

Why not just enforce the existing blood-pressure threshold more aggressively? Because the evidence says the threshold itself is miscalibrated and static targets cannot handle timing. The 31.8% versus 19% mortality split occurred under current threshold-based management, and the authors of that analysis explicitly conclude that the SBP ≥ 90 mmHg guideline may be too permissive, with hidden neuronal damage accumulating before alarms trigger (Early systemic insults). A fixed threshold is also a one-shot rule in what is a multi-step problem: it cannot reason about delayed drug effects, rebound, or the protracted two-week vulnerability window during which NfL and S100B stay elevated. Our decisive check is direct — if a simple tightened-threshold policy matches our learned policy's estimated value under noise, the complexity is unjustified and we will say so.

Why not just use standard propensity weighting to evaluate policies instead of doubly robust machinery? Because it demonstrably fails on this exact class of data. In the tranexamic acid replication, raw registry comparison showed 30% versus 15% mortality purely because sicker patients received the drug (median GCS 7 versus 13); standard IPW then estimated a spurious 9–10% harm, and only the doubly robust estimator with machine-learning imputation recovered the randomized-trial answer of −0.01 (Machine Learning Augmented Causal Inference). Since our entire claim rests on off-policy value estimates, we will run that paper's IPW-versus-DR comparison as a calibration exercise on our own estimators before trusting any policy ranking.

Why not just ignore execution noise and learn the optimal compliant policy? This is the most serious objection, and part of it is correct: in low-noise regimes a compliant optimal policy is fine. But the repeated-games literature shows that outcome-evaluating strategies without error absorption enter irreversible retaliation spirals at even 1% error, and the analogous clinical failure — a controller that interprets a missed titration and its resulting pressure dip as evidence its strategy failed, then oscillates — is exactly what crosstalk analysis predicts for interconnected systems (Achieving Sustainable Cooperation; Crosstalk in concurrent repeated games). Our experiment settles it empirically: we evaluate every policy, noise-trained or not, across the full error-rate sweep. If compliant-trained policies hold their value at all rates up to 40%, the noise-modeling component adds nothing and the ablation will reveal that honestly.

Why not just transfer the existing XGBoost mortality model to new regions and skip policies? Because prediction is not action. The XGBoost result is genuinely strong — external validation C-statistic 0.88 from the Chinese to the European registry, beating IMPACT at 0.80 and CRASH at 0.82 (Mortality Prediction) — and we reuse it as our state featurizer. But that paper itself flags that emergency interventions entered only as static covariates, leaving their causal efficacy unstudied. Static prediction tells you who will die; it cannot tell you what to do about it, nor whether a resuscitation policy learned in one referral system survives contact with another, which is precisely our open question.

Proposed Experiments and Decisive Checks

Experiment one is the noise sweep: train all policies at matched compute and tuning budget, evaluate with doubly robust off-policy estimation at injected error rates of 0, 5, 10, 20, and 40%. The decisive outcome is the shape of the degradation curves — our hypothesis predicts the compliant-trained and return-risk-averse variants fall off a cliff while the noise-trained, recovery-structured policy degrades gracefully, mirroring the cooperation-basin result in the Stag Hunt benchmark (The Price of Paranoia). Experiment two is the action-representation test: decoupled channels versus monolithic dosing, scored on estimated value net of toxicity penalties, with the penalty schedule grounded in the DEPO-versus-rhEPO hematocrit divergence and the TNFR1-selective sparing result (A novel neuroprotective derived peptide; Novel anti-TNFR1 domain antibody). Experiment three is the transfer test: policies trained on the China registry, evaluated on the European registry, reported alongside the XGBoost C-statistic of 0.88 as the static-prediction yardstick. We will additionally audit where transfer breaks by stratifying on referral and prehospital variables, since the mortality-prediction work shows prediction generalizes when protocols harmonize even while emergency practice stays geographically heterogeneous (Mortality Prediction). A finding that policies, unlike predictors, do not transfer would itself be a publishable negative result with direct regulatory relevance.

The honest scope limit: this work validates a policy-learning and evaluation methodology, not new pharmacology. The receptor-selectivity knowledge enters as action structure and reward shape, not as a clinical trial of any compound. What it would establish, for the first time, is whether forgiving, noise-aware control changes what offline RL recommends at the bedside — and whether the conservatism that makes these algorithms safe in sepsis makes them timid exactly where TBI patients can least afford it.