Emulate randomized clinical trials using heterogeneous treatment effect estimation for personalized treatments: Methodology review and benchmark

Yaobin Ling, Pulakesh Upadhyaya, Luyao Chen, Xiaoqian Jiang, Yejin KimJ. Biomed. Informatics 2023 · OpenReview

TL;DR

This paper tackles the challenge of estimating personalized treatment effects from observational healthcare data, a crucial step for developing precision medicine. By reviewing and benchmarking various machine learning methods for Heterogeneous Treatment Effect (HTE) estimation, the authors demonstrate that simpler models, like S-learners with Random Forests, can outperform complex deep learning approaches on sparse, real-world data, offering a more robust and interpretable path for emulating clinical trials.

What Problem Does This Paper Address?

In medicine, understanding the causal effect of a treatment is paramount, but randomized clinical trials (RCTs) are often expensive, time-consuming, or ethically challenging. Observational data (like electronic health records or insurance claims) offers a vast alternative, but it's plagued by confounding variables and selection bias, making it difficult to infer causation. While the average treatment effect (ATE) has been a traditional focus, it assumes a "one-size-fits-all" approach that doesn't account for individual patient variability. This is where Heterogeneous Treatment Effect (HTE) estimation comes in, aiming to predict how a treatment's effect varies across individuals or subgroups. However, applying these advanced HTE models to messy, real-world healthcare data has been limited. This paper bridges this gap by reviewing existing HTE methodologies and benchmarking them on a large-scale healthcare dataset to emulate an RCT, providing practical insights for their application in personalized medicine.

Methods

The authors provide a comprehensive review and comparison of HTE estimation methods, categorizing them into three main classes:

  • Double Machine Learning (DML): This approach uses machine learning to estimate nuisance parameters (like the probability of receiving treatment and the expected outcome) in a semi-parametric framework, allowing for more robust estimation of causal effects, especially in high-dimensional settings.
  • Meta-Learners: These are non-parametric methods that decompose the HTE estimation into several sub-problems, allowing the use of any arbitrary supervised machine learning model. Key meta-learners discussed include:
  • S-learner: Treats treatment assignment as just another feature.
  • T-learner: Fits separate models for treated and control groups.
  • X-learner: A variant of T-learner that uses additional steps to better handle imbalanced treatment groups.
  • Representation Learning: This class leverages deep neural networks to learn a "treatment-invariant" representation of patient features, aiming to balance distributions between treated and untreated groups and then estimate HTE.
  • Tree-based Methods: This includes causal trees and causal forests, which use recursive splitting criteria to identify subgroups with maximized treatment effect divergence.

The paper also details evaluation metrics like Robustness (ERMSE) and Estimated Goodness-of-Fit (IF-PEHE), which are crucial for assessing HTE models when the true effect is unobservable.

Datasets

The primary dataset used for the benchmark experiments is nationwide healthcare claim data from the Optum Clinformatics® Data Mart, covering January 2007 to June 2020. This dataset comprises approximately 6.5 billion claims with extensive diagnosis and medication codes. The authors used this data to emulate a specific clinical trial (NCT03991988) investigating the effect of Montelukast on Alzheimer's Disease (AD) risk.

Key Insights and Ideas

  • Simplicity can be powerful: Surprisingly, simpler models like the S-learner with a Random Forest base learner performed best on sparse and noisy healthcare claim data, outperforming more complex deep learning methods. This suggests that for certain real-world data challenges, a less complex approach can lead to more robust and accurate HTE estimation.
  • Shared structure matters: The success of the S-learner is attributed, in part, to its ability to learn a shared function for both treatment and control groups, which is beneficial when features are sparse and data is not rich. This implies that models that can capture underlying shared mechanisms between groups might be more effective.
  • Biologically zero treatment effect: When the true treatment effect is close to zero (as might be the case for Montelukast on AD, given current evidence), simpler models like the S-learner are known to be more accurate in estimating this zero effect.
  • Overfitting avoidance: The S-learner's smaller number of parameters (lower variance) may help it avoid overfitting compared to more complex meta-learners that require multiple prediction tasks.
  • Healthcare data limitations: Real-world healthcare administrative data is inherently incomplete, noisy, and often collected for billing rather than research, posing significant challenges for HTE estimation, particularly regarding unobserved confounding variables and data sparsity.

Key Results

The benchmark experiments revealed that:

  • S-learner with Random Forest achieved the best performance in terms of both Robustness (ERMSE = 0.0029) and Goodness-of-Fit (IF-PEHE = 9.82) among the evaluated models.
  • Other low-variance models like DML and DragonNet also showed good robustness (low ERMSE) but were not as competitive in goodness-of-fit compared to the best S-learner.
  • Representation learning methods, while having low ERMSE, generally had higher IF-PEHE, suggesting they might be less precise in predicting HTE on this specific dataset.
  • Feature importance analysis using Shapley scores identified age at baseline as a significant factor, indicating older patients may benefit less from leukotriene receptor antagonists for AD risk reduction. Conversely, patients with bone and cartilage disorders or gout showed a beneficial treatment effect.

Limitations and Future Work

The authors acknowledge several limitations:

  • Methodology Focus: The comparison was limited to non-parametric machine learning approaches, excluding parametric HTE estimation methods.
  • Time Zero Ambiguity: Due to short-term data, precisely separating pre-treatment and post-treatment comorbidities was challenging, potentially affecting treatment effect size estimation.
  • Unobserved Confounding: Healthcare administrative data often lacks crucial confounding variables like socioeconomic status, which can impact causal inference.
  • Data Sparsity and Multimodality: The data's sparsity and the need to integrate multimodal information (diagnoses, medications) present ongoing challenges.

Future work directions include:

  • Incorporating interactions between treatment and post-treatment covariates to better capture causal structure.
  • Developing HTE estimation methods that leverage similar mechanisms between treatments and placebos (e.g., active drugs with similar indications).
  • Infusing human knowledge (like PheWas codes) to complement incomplete healthcare records.
  • Incorporating temporal relationships of variables for more accurate HTE estimation.
  • Investigating HTE methods in clinical registries and trial data.

Summary updated

Abstract

Highlights•Heterogeneous treatment effect (HTE) models estimate the individual treatment effect.•Compare HTE models in terms of handling treatment assignment and selection bias.•Perform benchmark experiments by emulating clinical trials with insurance claim data.•Discuss lessons learned, future research direction, opportunity, and limitations.
BibTeX
@article{ling2023emulate,
  author = {Yaobin Ling and Pulakesh Upadhyaya and Luyao Chen and Xiaoqian Jiang and Yejin Kim},
  title = {{Emulate randomized clinical trials using heterogeneous treatment effect estimation for personalized treatments: Methodology review and benchmark}},
  journal = {J. Biomed. Informatics},
  year = {2023},
  url = {https://openreview.net/forum?id=wL1dYsc7C9},
}
Figure