Efficient and robust transfer learning of optimal individualized treatment regimes with right-censored survival data

Pan Zhao, Julie Josse, Shu YangarXiv 2023 · DOI · arXiv

TL;DR

This paper introduces a new framework for "transferring" treatment rules learned in clinical trials to real-world populations. By combining randomized controlled trial (RCT) data with observational studies, the authors developed a doubly robust method to estimate optimal individualized treatment regimes (ITRs) for survival data, even when the trial participants look very different from the general population.

What Problem Does This Paper Address?

In medicine, Randomized Controlled Trials (RCTs) are the gold standard for proving a treatment works. However, RCTs often have strict inclusion and exclusion criteria, meaning the people in the trial might not represent the "target population" (the actual patients doctors see in the clinic). This is known as covariate shift.

Furthermore, clinical outcomes are often "right-censored" survival data—meaning we know a patient survived at least until the end of the study, but we don't know exactly when the event of interest (like death) occurred.

The challenge is: How can we use high-quality RCT data to find the best treatment rule for a real-world population while accounting for both the population differences and the complexities of survival data?

Methods

The authors propose a transfer learning framework based on semiparametric theory. Their core contribution is an Augmented Calibration Weighting (ACW) estimator. Key components include:

  • Efficient Influence Function (EIF): They characterize the EIF for the "value function" (the expected outcome if a specific treatment rule is followed). This allows them to build an estimator that is statistically efficient.
  • Calibration Weighting (CW): Instead of trying to model the complex probability of a person being in an RCT versus an observational study, they use "entropy balancing." This forces the covariate distribution of the trial sample to match the target population's distribution.
  • Double Robustness: The method is "doubly robust," meaning it remains accurate if either the survival outcome model or the models for treatment assignment, censoring, and sampling are correctly specified.
  • Cross-fitting: To allow for modern, flexible machine learning (like Random Survival Forests), they use a cross-fitting procedure. This prevents "overfitting bias" and ensures that the final estimates follow a standard normal distribution for reliable hypothesis testing.

Datasets

The paper evaluates the method using:

  1. Extensive Simulations: Synthetic datasets designed to test scenarios where various models (outcome, sampling, or propensity) are intentionally misspecified.
  2. BICAR-ICU Trial: A multi-center RCT (387 patients) investigating sodium bicarbonate therapy for patients with severe metabolic acidaemia in the ICU.
  3. Observational Study (OS): A prospective study (193 patients) by Jung et al. (2011) used as the representative target population.

Key Insights and Ideas

  • RCTs aren't enough: A treatment rule that is "optimal" for the specific group of people in a clinical trial is often sub-optimal for the general population.
  • The Power of Calibration: Modeling the "sampling score" (who gets into a trial) is notoriously difficult and prone to error. By using calibration weighting to match moments (like means and variances) of patient characteristics, the researchers bypassed the need for a perfect sampling model.
  • Theoretical Breakthrough: The authors proved that the parameter indexing their treatment rule converges at a rate of $N^{-1/3}$. While this is slower than the standard $N^{-1/2}$ rate, they showed that the estimated value of the policy is still consistent and asymptotically normal, allowing for valid statistical inference (like confidence intervals).

Key Results

  • Simulation Success: In tests where the sampling model was wrong but the outcome model was right (and vice versa), the ACW estimator remained unbiased, while "naive" methods failed.
  • ICU Application: When applying the rule learned only from the RCT to the target population, the estimated Restricted Mean Survival Time (RMST) was 15.37 days.
  • Transfer Learning Improvement: Using the proposed ACW method to "transfer" the knowledge to the target population resulted in a significantly better treatment rule, with an RMST of 19.52 days (95% CI: [17.74, 21.30]). This suggests that the transfer learning approach could lead to nearly 4 additional days of survival within a 28-day window for these critical patients.

Limitations and Future Work

  • Linear Rules: The current work focuses primarily on linear decision rules (e.g., "Treat if $AX + B > 0$"). The authors suggest that extending this to more complex structures, like tree-based or list-based regimes, would be a valuable next step.
  • Multiple Decision Points: This paper looks at a single treatment decision. Real-world medical care often involves a sequence of decisions over time (Dynamic Treatment Regimes), which adds significant complexity regarding censoring and timing.
  • Assumptions: The method relies on "survival mean exchangeability"—the assumption that the relationship between patient characteristics and outcomes is the same in both the trial and the real world. If there are unmeasured factors that differ between the two, the results could be biased.

Summary updated

Abstract

An individualized treatment regime (ITR) is a decision rule that assigns treatments based on patients' characteristics. The value function of an ITR is the expected outcome in a counterfactual world had this ITR been implemented. Recently, there has been increasing interest in combining heterogeneous data sources, such as leveraging the complementary features of randomized controlled trial (RCT) data and a large observational study (OS). Usually, a covariate shift exists between the source and target population, rendering the source-optimal ITR unnecessarily optimal for the target population. We present an efficient and robust transfer learning framework for estimating the optimal ITR with right-censored survival data that generalizes well to the target population. The value function accommodates a broad class of functionals of survival distributions, including survival probabilities and restrictive mean survival times (RMSTs). We propose a doubly robust estimator of the value function, and the optimal ITR is learned by maximizing the value function within a pre-specified class of ITRs. We establish the $N^{-1/3}$ rate of convergence for the estimated parameter indexing the optimal ITR, and show that the proposed optimal value estimator is consistent and asymptotically normal even with flexible machine learning methods for nuisance parameter estimation. We evaluate the empirical performance of the proposed method by simulation studies and a real data application of sodium bicarbonate therapy for patients with severe metabolic acidaemia in the intensive care unit (ICU), combining a RCT and an observational study with heterogeneity.
BibTeX
@misc{zhao2023efficient,
  author = {Pan Zhao and Julie Josse and Shu Yang},
  title = {{Efficient and robust transfer learning of optimal individualized treatment regimes with right-censored survival data}},
  year = {2023},
  eprint = {2301.05491},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.05491},
}
Figure