Supervised variable selection in randomised controlled trials prior to exploration of treatment effect heterogeneity: an example from severe malaria

Chieh‐Hsi Wu, Chris HolmesarXiv 2019 · DOI · arXiv

Abstract

Exploration of treatment effect heterogeneity (TEH) is essential for modern stratified medicine, yet analysts often face a "curse of dimensionality." Including too many variables in interaction discovery can dilute the statistical power to detect true signals. While current guidance favors unsupervised methods like PCA, this paper demonstrates that supervised variable selection—using outcome-driven dimension reduction—can maintain power without inflating false-positive rates. We apply this framework to a landmark severe malaria trial, identifying key drivers of treatment response.

Introduction

In clinical research, the "average treatment effect" often masks critical differences in how individual patients respond to therapy. Identifying these differences, known as Treatment Effect Heterogeneity (TEH), is the cornerstone of precision medicine. However, as the number of baseline patient characteristics (covariates) grows, researchers encounter a significant statistical hurdle: the multiple testing problem.

Testing for interactions between a treatment and dozens of covariates significantly reduces the power to detect any single true interaction due to the necessary stringency of multiple-testing corrections. Traditional solutions involve unsupervised dimension reduction, such as Principal Component Analysis (PCA), which reduces the variable space without looking at the patient outcomes. The risk, however, is that unsupervised methods might discard the very variables most relevant to the treatment's efficacy.

The Two-Stage Supervised Framework

Wu and Holmes propose a two-stage framework that leverages supervised learning to prioritize variables before testing for interactions.

Stage 1: Prognostic Screening

In the first stage, the entire dataset is used to rank variables based on their "main effect"—their ability to predict the clinical outcome regardless of the treatment assigned. Because prognostic variables (those that strongly predict outcome) are frequently the same variables that modify treatment effects, this screening focuses the analysis on the most promising candidates. Researchers can use flexible machine learning tools, such as LASSO or Gradient Boosting Machines (GBM), to perform this ranking.

Stage 2: Interaction Testing

In the second stage, a formal hypothesis test (e.g., a Likelihood Ratio Test) is performed on a reduced set of variables—either the top $K$ variables or their principal components. This stage specifically tests for interactions between the treatment and these selected covariates.

Theoretical Foundation: The Independence Property

The validity of this approach hinges on a critical mathematical insight. In a randomized controlled trial (RCT), the estimate of a variable’s main effect is statistically independent of the estimate of its interaction effect under the null hypothesis.

This independence (Theorem 1 in the paper) ensures that using the outcome to select variables in Stage 1 does not "pollute" or bias the p-values generated in Stage 2. This is a unique advantage of randomized data; in observational studies, this independence does not hold, and supervised selection would likely lead to inflated Type-I error rates.

Application: Severe Malaria and the SEAQUAMAT Trial

The authors demonstrated their framework using data from the SEAQUAMAT trial, which compared intravenous quinine and artesunate in 1,461 patients with severe malaria. The dataset included 14 biological baseline measurements, such as coma score, age, heart rate, and jaundice.

Findings

When researchers applied a "naive" approach—testing all 14 variables for interactions simultaneously—the result was statistically non-significant ($p = 0.115$). However, by applying supervised screening to focus on the top 7 components, the framework revealed a significant interaction ($p = 0.0435$).

Further analysis identified jaundice and systolic blood pressure as the primary drivers of this heterogeneity. This suggests that the benefit of artesunate over quinine may vary significantly depending on these specific clinical markers.

Comparison with Internal Risk Models

A common existing strategy for TEH is the "internal risk model," where a single risk score is calculated for each patient, and researchers test if the treatment effect varies by risk level. Wu and Holmes show that their method is a generalized version of this approach. While risk models collapse all information into one dimension, the supervised framework allows for a more nuanced exploration of which specific factors drive the treatment interaction.

Conclusion and Limitations

Supervised variable selection provides a powerful tool for clinical trialists to discover subgroup effects without the "fishing expedition" risks typically associated with high-dimensional data.

However, some caveats remain. The primary test is "global," meaning it identifies the presence of an interaction within a group of variables rather than providing individual p-values for each covariate. Therefore, identifying specific drivers should be viewed as hypothesis-generating. Furthermore, the number of variables to retain ($K$) should ideally be pre-specified to maintain the highest level of rigor. By following this structured approach, researchers can more effectively bridge the gap between "average" trial results and personalized patient care.

Summary updated

Abstract

Exploration of treatment effect heterogeneity (TEH) is an increasingly important aspect of modern statistical analysis for stratified medicine in randomised controlled trials (RCTs) as we start to gather more information on trial participants and wish to maximise the opportunities for learning from data. However, the analyst should refrain from including a large number of variables in a treatment interaction discovery stage. Because doing so can significantly dilute the power to detect any true outcome-predictive interactions between treatments and covariates. Current guidance is limited and mainly relies on the use of unsupervised learning methods, such as hierarchical clustering or principal components analysis, to reduce the dimension of the variable space prior to interaction tests. In this article we show that outcome-driven dimension reduction, i.e. supervised variable selection, can maintain power without inflating the type-I error or false-positive rate. We provide theoretical and applied results to support our approach. The applied results are obtained from illustrating our framework on the dataset from an RCT in severe malaria. We also pay particular attention to the internal risk model approach for TEH discovery, which we show is a particular case of our method and we point to improvements over current implementation.
BibTeX
@misc{wu2019supervised,
  author = {Chieh‐Hsi Wu and Chris Holmes},
  title = {{Supervised variable selection in randomised controlled trials prior to exploration of treatment effect heterogeneity: an example from severe malaria}},
  year = {2019},
  eprint = {1901.03531},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1901.03531},
}
Figure