Constructing a T ‐Test for Value Function Comparison of Individualized Treatment Regimes in the Presence of Multiple Imputation for Missing Data

Minxin Lu, Annie Green Howard, Penny Gordon‐Larsen, Katie A. Meyer, Hsiao‐Chuan Tien, Shufa Du, Huijun Wang, Bing Zhang, Michael R. Kosorok

TL;DR

This paper introduces a computationally efficient t-test to compare the performance of different personalized treatment rules (Individualized Treatment Regimes), even when dealing with missing data. By leveraging the asymptotic normality of value functions, the authors provide a way to generate valid p-values and confidence intervals without the heavy computational burden of traditional resampling methods like bootstrapping.

What Problem Does This Paper Address?

In precision medicine, researchers develop Individualized Treatment Regimes (ITRs)—rules that recommend specific treatments based on a patient's unique characteristics (like genetics, lifestyle, or medical history). To see if one rule is actually better than another (or better than a "one-size-fits-all" approach), we use a metric called the Value Function, which represents the expected outcome if the entire population followed that rule.

The challenge lies in statistical inference. Comparing two value functions is notoriously difficult because their distributions are complex. Existing solutions like bootstrapping or jackknifing are "brute-force" methods that require re-running models hundreds or thousands of times. This becomes a massive computational bottleneck when datasets are large or when data is missing. Missing data requires Multiple Imputation (MI)—creating several versions of the dataset to account for uncertainty—which multiplies the computational cost of bootstrapping even further.

Methods

The authors propose a t-test based method applied to a held-out test set. The core idea relies on the mathematical proof that, in large samples, the estimated value function follows a normal distribution (asymptotic normality).

Key components of the approach include:

  • Training/Test Split: The treatment rules are learned on a training set and evaluated on an independent test set to ensure unbiased results.
  • Propensity Score Integration: For observational data, the method accounts for the uncertainty in estimating the "propensity score" (the probability of receiving a treatment given certain covariates).
  • Multiple Imputation Extension: The authors adapt their variance estimator to work with multiple imputed datasets. They provide a formula to combine the "within-imputation" variance and the "between-imputation" variance, similar to Rubin’s Rules but tailored for value functions.
  • Closed-form Variance: Unlike the bootstrap, which estimates variance through repeated sampling, this method uses a direct formula derived from the "influence function" of the value estimator.

Datasets

The researchers evaluated their method using:

  1. Simulated Data: Four scenarios designed to mimic randomized controlled trials and observational studies, both with complete data and data "Missing at Random" (MAR).
  2. China Health and Nutrition Survey (CHNS): A real-world observational dataset. They specifically looked at the 2009 study year (~5,241 individuals) to compare different strategies for managing hypertension through physical activity (Moderate-to-Vigorous Physical Activity, or MVPA).

Key Insights and Ideas

  • Resampling isn't always necessary: The "aha moment" is that we don't need to simulate the distribution of the value function if we can mathematically describe its limiting behavior. This allows for a "plug-and-play" t-test that is nearly as accurate as a bootstrap but orders of magnitude faster.
  • Handling the "Missingness" Uncertainty: The method doesn't just ignore missing data; it explicitly incorporates the extra layer of uncertainty introduced by the imputation process into the final p-value.
  • Model Agnostic: The test can be used to compare rules generated by any method (e.g., Q-learning, D-learning, or even simple clinical guidelines) as long as they are evaluated on a test set.

Key Results

  • Simulation Accuracy: In head-to-head tests against the bootstrap, the proposed t-test achieved nearly identical Type I error rates and statistical power.
  • Computational Efficiency: While the bootstrap requires 500+ iterations, the t-test requires only one pass through the data (per imputation), making it highly scalable for "Big Data" in healthcare.
  • CHNS Application: When applying the method to the China Health and Nutrition Survey, they compared a Q-learning-based personalized rule to a universal "high physical activity" recommendation. While the personalized rule had a slightly better (lower) predicted risk of hypertension, the t-test revealed the difference was not statistically significant ($p = 0.156$). This suggests that for this specific population, a personalized approach might achieve similar health outcomes to a high-activity mandate, potentially offering more flexibility for patients.

Limitations and Future Work

  • Propensity Model Sensitivity: The method's validity relies on the correct specification of the propensity score model. If the model used to account for treatment assignment bias is wrong, the value function estimates (and the test) will be biased.
  • Extreme Propensity Scores: In observational data, if some individuals have a near-zero or near-one probability of receiving a treatment, the variance estimates can become unstable.
  • Future Directions: The authors suggest extending this framework to multi-stage ITRs (where treatments are adjusted over time) and developing more robust estimators (like doubly robust methods) to protect against model misspecification. They also aim to expand the test to compare more than two treatment regimes simultaneously.

Summary updated

Abstract

Optimal individualized treatment decision-making has improved health outcomes in recent years. The value function is commonly used to evaluate the goodness of an individualized treatment decision rule. Despite recent advances, comparing value functions between different treatment decision rules or constructing confidence intervals around value functions remains difficult. We propose a t-test based method applied to a test set that generates valid p-values to compare value functions between a given pair of treatment decision rules when some of the data are missing. We demonstrate the ease in use of this method and evaluate its performance via simulation studies and apply it to the China Health and Nutrition Survey data.
BibTeX
@inproceedings{lu2025constructing,
  author = {Minxin Lu and Annie Green Howard and Penny Gordon‐Larsen and Katie A. Meyer and Hsiao‐Chuan Tien and Shufa Du and Huijun Wang and Bing Zhang and Michael R. Kosorok},
  title = {{Constructing a <i>T</i> ‐Test for Value Function Comparison of Individualized Treatment Regimes in the Presence of Multiple Imputation for Missing Data}},
  booktitle = {Statistics},
  year = {2025},
  doi = {10.1002/sim.70210},
}

Loading related content...

Figure