Doubly robust estimators for generalizing treatment effects on survival outcomes from randomized controlled trials to a target population

Dasom Lee, Shu Yang, Xiaofei Wang

TL;DR

Randomized Controlled Trials (RCTs) are the gold standard for medical evidence, but their results often don't apply to the general population because trial participants are "cherry-picked." This paper introduces a new "doubly robust" statistical framework that combines small, biased RCT data with large, representative observational studies to accurately estimate survival benefits for real-world patients.

What Problem Does This Paper Address?

In medicine, we rely on Randomized Controlled Trials (RCTs) to tell us if a treatment works. However, RCTs have a "generalizability" problem. Because trials often have strict inclusion and exclusion criteria, the people in the study might be younger, healthier, or otherwise different from the "average" patient seen in a clinic. If a treatment's effect depends on these characteristics (heterogeneity), the trial's findings might be biased when applied to the general population.

This is particularly tricky for survival outcomes (time-to-event data). Standard methods often struggle with "censoring" (when a patient leaves a study before an event occurs) and the fact that the relationship between a treatment and survival can change over time. Existing methods to fix this usually rely on a single statistical model; if that model is even slightly wrong, the final estimate can be wildly off.

Methods

The authors propose a semiparametric framework called Augmented Calibration Weighting (ACW). The core innovation lies in its Double Robustness. To estimate the treatment effect, the method uses two different models:

  1. An Outcome Model: Predicts survival based on patient characteristics.
  2. A Weighting Model: Predicts the probability of a patient being in the RCT versus the general population.

The "magic" of double robustness is that the final estimate remains accurate (consistent) if either of these models is correctly specified. If both are correct, the estimator is "locally efficient," meaning it has the lowest possible statistical uncertainty.

To make the method even more flexible, the authors use the Method of Sieves. Instead of assuming a rigid mathematical shape for their models (like a simple linear relationship), they use a series of basis functions that can adapt to the true complexity of the data. This provides "rate-double robustness," allowing for accurate results even if the underlying models are complex and difficult to estimate perfectly.

Datasets

The researchers validated their approach using two primary sources:

  • Simulation Studies: They generated a synthetic target population of 200,000 individuals to test how the estimator handles various scenarios where models are intentionally "broken" or misspecified.
  • Real-World Application (Lung Cancer): They applied the method to data from the CALGB 9633 trial (an RCT with 319 patients) and the National Cancer Database (NCDB) (an observational study with over 15,000 patients). The goal was to see if adjuvant chemotherapy actually helps patients with early-stage resected non-small cell lung cancer (NSCLC) in the real world.

Key Insights and Ideas

  • Leveraging "Outcome-Free" Data: Interestingly, you don't necessarily need the survival outcomes from the large observational study. You just need the covariates (patient characteristics) of that study to understand what the "target population" looks like.
  • Moving Beyond Hazard Ratios: While doctors love Hazard Ratios, they can be misleading if the treatment's effect changes over time. This paper focuses on Restricted Mean Survival Time (RMST)—essentially asking, "On average, how much longer does a patient live over a 12-year window?" This is much more intuitive for clinical decision-making.
  • The Power of Integration: The paper demonstrates that "data integration" isn't just about making a dataset bigger; it's about using the strengths of one data source (the randomization of the RCT) to fix the weaknesses of another (the lack of representativeness).

Key Results

  • Superior Robustness: In simulations, when the statistical models were intentionally misspecified, the authors' ACW estimator remained unbiased, while "Naive" (RCT-only) and "IPSW" (weighting-only) methods failed.
  • A Clinical "Aha!" Moment: The original CALGB 9633 trial concluded that chemotherapy did not significantly improve 12-year survival for these lung cancer patients. However, the trial population was younger and healthier than the general population.
  • Real-World Benefit Found: When the authors "transported" the trial results to the real-world NCDB population using their new estimator, they found a significant ~1-year increase in average survival time for patients receiving chemotherapy. This suggests the treatment is more effective in the broader, older, and higher-risk real-world population than the trial originally suggested.

Limitations and Future Work

  • The "Ignorability" Assumption: The method assumes that researchers have measured all the important variables that influence both trial participation and survival. If there's a "hidden" factor that makes people join trials AND affects their survival, the results could still be biased.
  • Sensitivity Analysis: The authors suggest that future work should include "sensitivity analyses" to determine how much a hidden, unmeasured factor would have to influence the data to flip the results.
  • Individualized Medicine: While this paper looks at the average effect for a population, the authors note that the next step is using these integrated datasets to predict which specific individuals will benefit most from a treatment.

Summary updated

Abstract

In the presence of heterogeneity between the randomized controlled trial (RCT) participants and the target population, evaluating the treatment effect solely based on the RCT often leads to biased quantification of the real-world treatment effect. To address the problem of lack of generalizability for the treatment effect estimated by the RCT sample, we leverage observational studies with large samples that are representative of the target population. This paper concerns evaluating treatment effects on survival outcomes for a target population and considers a broad class of estimands that are functionals of treatment-specific survival functions, including differences in survival probability and restricted mean survival times. Motivated by two intuitive but distinct approaches, i.e., imputation based on survival outcome regression and weighting based on inverse probability of sampling, censoring, and treatment assignment, we propose a semiparametric estimator through the guidance of the efficient influence function. The proposed estimator is doubly robust in the sense that it is consistent for the target population estimands if either the survival model or the weighting model is correctly specified, and is locally efficient when both are correct. In addition, as an alternative to parametric estimation, we employ the nonparametric method of sieves for flexible and robust estimation of the nuisance functions and show that the resulting estimator retains the root-n consistency and efficiency, the so-called it rate-double robustness. Simulation studies confirm the theoretical properties of the proposed estimator and show it outperforms competitors. We apply the proposed method to estimate the effect of adjuvant chemotherapy on survival in patients with early-stage resected non-small lung cancer.
BibTeX
@article{lee2022doubly,
  author = {Dasom Lee and Shu Yang and Xiaofei Wang},
  title = {{Doubly robust estimators for generalizing treatment effects on survival outcomes from randomized controlled trials to a target population}},
  journal = {Journal of Causal Inference},
  year = {2022},
  doi = {10.1515/jci-2022-0004},
}
Figure