Probing the Effective Treatment Thresholds for Alteplase in Acute Ischemic Stroke With Regression Discontinuity Designs

Andrew M. Naidech, Patrick N. Lawlor, Haolin Xu, Gregg C. Fonarow, Ying Xian, Eric E. Smith, Lee H. Schwamm, Roland Matsouaka, Shyam Prabhakaran, Ioana Marinescu, Konrad P. KördingFrontiers in Neurology 2020 · DOI

Abstract

Randomized Controlled Trials (RCTs) are considered the gold standard for measuring the efficacy of medical interventions. However, RCTs are expensive, and use a limited population. Techniques to estimate the effects of stroke interventions from observational data that minimize confounding would be useful. We used regression discontinuity design (RDD), a technique well-established in economics, on the Get With The Guidelines-Stroke (GWTG-Stroke) data set. RDD, based on regression, measures the occurrence of a discontinuity in an outcome (e.g., odds of home discharge) as a function of an intervention (e.g., alteplase) that becomes significantly more likely when crossing the threshold of a continuous variable that determines that intervention (e.g., time from symptom onset, since alteplase is only given if symptom onset is less than e.g., 3 h). The technique assumes that patients near either side of a threshold (e.g., 2.99 and 3.01 h from symptom onset) are indistinguishable other than the use of the treatment. We compared outcomes of patients whose estimated onset to treatment time fell on either side of the treatment threshold for three cohorts of patients in the GWTG-Stroke data set. This data set spanned three different treatment thresholds for alteplase (3 h, 2003-2007, N = 1,869; 3 h, 2009-2016, N = 13,086, and 4.5 h, 2009-2016, N = 6,550). Patient demographic characteristics were overall similar across the treatment thresholds. We did not find evidence of a discontinuity in clinical outcome at any treatment threshold attributable to alteplase. Potential reasons for failing to find an effect include violation of some RDD assumptions in clinical care, large sample sizes required, or already-well-chosen treatment threshold.

Introduction

In acute ischemic stroke, "time is brain." Alteplase, a "clot-busting" drug, is highly effective but carries a risk of brain hemorrhage, so its use is strictly limited to specific time windows—traditionally 3 or 4.5 hours from symptom onset. These windows were established by Randomized Controlled Trials (RCTs), which are the gold standard but are incredibly expensive and slow to conduct.

This paper explores a clever "quasi-experimental" approach to this classic problem: using Regression Discontinuity Design (RDD)—a method popular in economics for evaluating policy changes—on existing observational data to verify these thresholds. If a patient arrives at 2 hours and 59 minutes, they get the drug; at 3 hours and 1 minute, they likely do not. Since these two patients are clinically almost identical, any "jump" in their recovery outcomes can be attributed to the drug itself, effectively mimicking a randomized trial.

Methodology

The study utilized two primary variations of RDD:

  • Sharp RDD: Assumes the treatment threshold is a hard rule. If you are below the cutoff, you get the drug; if above, you don't.
  • Fuzzy RDD: More realistic for clinical settings, this accounts for "non-compliance"—cases where a doctor might give the drug slightly late or withhold it slightly early based on clinical judgment. This uses a two-stage instrumental variable approach to model the probability of treatment.

A unique challenge was the Running Variable Imputation. For patients who did not receive the drug, there is no "time to treatment" recorded. The authors estimated this by using the median time from clinical imaging to treatment at each specific hospital to create a comparable "estimated onset to treatment time" (OTT) for all patients.

Study Cohorts

The study analyzed data from the Get With The Guidelines-Stroke (GWTG-Stroke) registry, focusing on three cohorts defined by their treatment windows and time periods.

Table 1: Baseline characteristics of the three study cohorts from the GWTG-Stroke registry.

As shown in the baseline characteristics, patient demographics were largely similar across the treatment thresholds, supporting the RDD assumption that patients near the cutoff are comparable.

Key Results

Across all three cohorts and both Sharp and Fuzzy models, there was no strong evidence of a discontinuity in clinical outcomes—such as good discharge disposition or ambulatory status—attributable to Alteplase at the 3-hour or 4.5-hour marks.

Table 2: Sharp and Fuzzy RDD results for Cohort C (3-hour threshold, post-2009), showing no significant discontinuity in most clinical outcomes.

While the "Fuzzy RDD" confirmed that the probability of receiving Alteplase did indeed jump significantly at the thresholds (proving the "instrument" was strong), the outcome change was not significant. In some models, the researchers actually saw "weak effects" in the wrong direction (e.g., treated patients having slightly longer hospital stays), likely due to the increased monitoring required for those receiving Alteplase.

Interpreting the "Null" Findings

The authors suggest several reasons why a clear discontinuity was not detected despite the known efficacy of Alteplase:

  • Linear Decay: The benefit of Alteplase likely declines linearly with every passing minute. A "discontinuity" only appears if there is a sudden drop-off in efficacy, which might not exist.
  • Clinical Selection: Doctors may be more likely to treat a "borderline" patient who looks like they will recover well, which "blurs" the threshold and violates the random-like distribution RDD requires.
  • Power Requirements: RDD is statistically "expensive." It often requires much larger sample sizes than RCTs to detect the same effect size because it only looks at the narrow slice of data right at the threshold.

Limitations and Future Work

The study identified several hurdles for using RDD in medicine. First, the outcomes were limited to hospital discharge data; Alteplase effects are often most visible at the 3-month mark. Second, missing data regarding stroke severity (NIH Stroke Scale) acted as a critical confounder. Finally, estimating treatment time for untreated patients adds "noise" to the model.

Future RDD studies should leverage even larger datasets and focus on long-term functional outcomes, such as the modified Rankin Scale at 90 days, to better capture the nuances of stroke recovery.

Summary updated

Abstract

Randomized Controlled Trials (RCTs) are considered the gold standard for measuring the efficacy of medical interventions. However, RCTs are expensive, and use a limited population. Techniques to estimate the effects of stroke interventions from observational data that minimize confounding would be useful. We used regression discontinuity design (RDD), a technique well-established in economics, on the Get With The Guidelines-Stroke (GWTG-Stroke) data set. RDD, based on regression, measures the occurrence of a discontinuity in an outcome (e.g., odds of home discharge) as a function of an intervention (e.g., alteplase) that becomes significantly more likely when crossing the threshold of a continuous variable that determines that intervention (e.g., time from symptom onset, since alteplase is only given if symptom onset is less than e.g., 3 h). The technique assumes that patients near either side of a threshold (e.g., 2.99 and 3.01 h from symptom onset) are indistinguishable other than the use of the treatment. We compared outcomes of patients whose estimated onset to treatment time fell on either side of the treatment threshold for three cohorts of patients in the GWTG-Stroke data set. This data set spanned three different treatment thresholds for alteplase (3 h, 2003-2007, <i>N</i> = 1,869; 3 h, 2009-2016, <i>N</i> = 13,086, and 4.5 h, 2009-2016, <i>N</i> = 6,550). Patient demographic characteristics were overall similar across the treatment thresholds. We did not find evidence of a discontinuity in clinical outcome at any treatment threshold attributable to alteplase. Potential reasons for failing to find an effect include violation of some RDD assumptions in clinical care, large sample sizes required, or already-well-chosen treatment threshold.
BibTeX
@inproceedings{naidech2020probing,
  author = {Andrew M. Naidech and Patrick N. Lawlor and Haolin Xu and Gregg C. Fonarow and Ying Xian and Eric E. Smith and Lee H. Schwamm and Roland Matsouaka and Shyam Prabhakaran and Ioana Marinescu and Konrad P. Körding},
  title = {{Probing the Effective Treatment Thresholds for Alteplase in Acute Ischemic Stroke With Regression Discontinuity Designs}},
  booktitle = {Frontiers in Neurology},
  year = {2020},
  doi = {10.3389/fneur.2020.00961},
}
Figure