Technical Background for "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis"

Xiaotong Jiang, Amanda E. Nelson, Rebecca J. Cleveland, Daniel P. Beavers, Todd A. Schwartz, Liubov Arbeeva, Carolina Álvarez, Leigh F. Callahan, Stephen P. Messier, Richard F. Loeser, Michael R. Kosorok

Introduction

In traditional clinical research, the primary goal is often to identify the treatment that performs best on average across a population. However, precision medicine shifts this focus toward Individualized Treatment Rules (ITRs)—mathematical frameworks that map specific patient characteristics, such as biomarkers and clinical history, to the most effective intervention for that individual.

This report provides the statistical foundation for the methodology used in Jiang et al. (2020), which developed optimal exercise and weight loss treatments for overweight and obese adults with knee osteoarthritis. While the primary clinical study focuses on patient outcomes, this technical companion elaborates on the use of the jackknife method (leave-one-out cross-validation) to estimate the performance and variance of these precision medicine models.

The Challenge: Estimating the Value Function

To determine if an ITR is superior to a "one-size-fits-all" approach, researchers must estimate its Value Function. The Value Function represents the expected outcome if a specific decision rule were applied to the entire population.

Validating these functions in clinical trials is challenging due to limited sample sizes. Standard techniques like k-fold cross-validation can introduce noise or bias because they require setting aside significant portions of data for testing, leaving less data available for the model to learn the optimal rule. There is a critical need for a stable, mathematically justified method to estimate both the Value and the associated uncertainty (variance) when data is scarce.

Methodology: The Jackknife Framework

The authors propose the Jackknife Estimator ($V^{jk}$) as a robust alternative for clinical validation. The process involves:

  1. ITR Estimation: Utilizing both indirect (regression-based) and direct (classification-based, such as Outcome Weighted Learning) approaches to identify the optimal rule $d^{opt}$.
  2. The Jackknife Value Estimator: Defining a weighted estimator where the decision rule for each participant is trained on the remaining $n-1$ participants. This maximizes data efficiency by using nearly the entire dataset for training in every iteration.
  3. Variance Derivation: A significant contribution of this work is the derivation of a variance estimator inspired by influence functions. This allows for the calculation of standard errors and the construction of confidence intervals, which are essential for determining the statistical significance of the ITR's performance.
  4. Asymptotic Theory: The report provides formal proofs of consistency. It demonstrates that as the sample size increases, the jackknife estimate converges to the true value, and its variance predictably diminishes.

Simulation and Validation

To validate the mathematical framework, the authors conducted simulations across four synthetic scenarios with varying decision boundaries:

  • Scenario 1: Highly non-linear concentric circles.
  • Scenario 2: Linear but discontinuous nested steps.
  • Scenario 3: Simple linear parallel diagonal lines.
  • Scenario 4: Non-linear nested parabolas.

These scenarios tested the model's ability to recover the optimal rule under different geometric complexities. Additionally, the methodology was applied to data from the Intensive Diet and Exercise for Arthritis (IDEA) trial, a randomized clinical trial studying the effects of exercise (E), dietary weight loss (D), and a combination (D+E) on overweight adults with knee osteoarthritis.

Key Results

Statistical Power and Coverage

The simulations revealed that the jackknife method is highly reliable for uncertainty estimation. For a sample size of $n=800$, the 95% confidence intervals achieved approximately 96% coverage across most scenarios. Furthermore, the statistical power to detect differences between a Precision Medicine Model (PMM) and a Zero-Order Model (ZOM—the standard "average" treatment) increased significantly with sample size. In Scenario 3, for instance, power rose from 15% at $n=50$ to 94% at $n=800$.

Normality and Bias Correction

Q-Q plots and Shapiro-Wilk tests confirmed that the test statistics follow a standard normal distribution as the sample size grows, justifying the use of Z-tests for comparing treatment strategies. The authors also implemented a "bias-corrected" influence-function-based estimator ($R_i^{jk}$), ensuring that the sum of influence components equals zero, which stabilizes the variance calculation.

Discussion and Limitations

While the jackknife method is data-efficient and mathematically rigorous, the authors note several considerations for its application:

  • Sample Size Requirements: When $n \le 200$, the statistical power to prove that a precision medicine approach outperforms a standard approach remains relatively low, even if the precision model is more accurate.
  • Boundary Complexity: The framework performs best with linear boundaries; it faces greater challenges with highly complex, non-linear boundaries like those in Scenario 1.
  • Computational Intensity: Because the jackknife requires training the model $n$ times, it may be computationally demanding for very large datasets or highly complex machine learning architectures.

Conclusion

This technical report establishes the jackknife method as a mathematically sound and data-efficient tool for validating individualized treatment rules in clinical settings. By providing a framework for estimating both the performance and the variance of ITRs, it enables researchers to move beyond "average" treatment effects toward more personalized, evidence-based care for conditions like knee osteoarthritis.

Summary updated

Abstract

We provide additional statistical background for the methodology developed in the clinical analysis of knee osteoarthritis in "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis" (Jiang et al. 2020). Jiang et al. 2020 proposed a pipeline to learn optimal treatment rules with precision medicine models and compared them with zero-order models with a Z-test. The model performance was based on value functions, a scalar that predicts the future reward of each decision rule. The jackknife (i.e., leave-one-out cross validation) method was applied to estimate the value function and its variance of several outcomes in IDEA. IDEA is a randomized clinical trial studying three interventions (exercise (E), dietary weight loss (D), and D+E) on overweight and obese participants with knee osteoarthritis. In this report, we expand the discussion and justification with additional statistical background. We elaborate more on the background of precision medicine, the derivation of the jackknife estimator of value function and its estimated variance, the consistency property of jackknife estimator, as well as additional simulation results that reflect more of the performance of jackknife estimators. We recommend reading Jiang et al. 2020 for clinical application and interpretation of the optimal ITR of knee osteoarthritis as well as the overall understanding of the pipeline and recommend using this article to understand the underlying statistical derivation and methodology.
BibTeX
@misc{jiang2020technical,
  author = {Xiaotong Jiang and Amanda E. Nelson and Rebecca J. Cleveland and Daniel P. Beavers and Todd A. Schwartz and Liubov Arbeeva and Carolina Álvarez and Leigh F. Callahan and Stephen P. Messier and Richard F. Loeser and Michael R. Kosorok},
  title = {{Technical Background for "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis"}},
  year = {2020},
  eprint = {2001.09930},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2001.09930},
}
Figure