An efficacy evaluation method for non-normal outcomes in randomized controlled trials

Yang Li, Zhang Zhang, Qian Feng, Danhui Yi, Fang LüScientific 2019 · DOI

TL;DR

This paper introduces a novel Exponential Squared Loss (ESL) regression model for analyzing randomized controlled trials (RCTs) where outcome residuals are not normally distributed. The ESL method offers more robust and efficient parameter estimation compared to traditional Ordinary Least Squares (OLS) regression, especially when dealing with contaminated or non-normal data, making it a valuable tool for real-world clinical studies.

What Problem Does This Paper Address?

Randomized controlled trials (RCTs) are the gold standard for evaluating treatment efficacy. A common statistical approach is linear regression, which uses Ordinary Least Squares (OLS) to estimate treatment effects while accounting for confounding variables. However, OLS relies on the assumption that the model's residuals (the differences between observed and predicted values) are normally distributed. When this assumption is violated – a frequent occurrence in clinical data due to measurement errors, mixed patient populations, or other factors – OLS becomes inefficient, leading to less reliable parameter estimates. This paper addresses this critical limitation by proposing a robust alternative that performs well even with non-normal residuals.

Methods

The core contribution of this paper is the introduction and application of an Exponential Squared Loss (ESL) regression model. Unlike OLS, which minimizes the sum of squared residuals, ESL aims to maximize a function that penalizes deviations less severely for "normal" data points but more significantly for outliers or non-normal data. The ESL function is defined as $\Phi(t) = 1 - \exp(-\frac{t^2}{\gamma})$, where $t$ represents a scaled residual and $\gamma$ is a tuning parameter.

The authors adapt a robust estimation procedure, inspired by previous work on variable selection, to estimate the model parameters ($\beta$). This procedure involves iteratively:

  1. Identifying pseudo-outliers: A set of data points with large residuals is identified.
  2. Updating the tuning parameter ($\gamma$): A data-driven approach is used to select $\gamma$ that balances robustness and efficiency.
  3. Updating regression coefficients ($\beta$): The parameters are re-estimated using the ESL objective function.

This iterative process, starting with an initial OLS estimate, continues until convergence, providing a robust estimate of the treatment effect.

Datasets

The primary dataset used for evaluation is from a phase III randomized clinical trial investigating the efficacy of a traditional Chinese medicine for mild cognitive impairment (MCI). The study involved 216 patients in the treatment arm and 108 in the control arm. The outcome variable analyzed was the difference in Alzheimer's Disease Assessment Scale-Cognitive (ADAS-cog) scores between the final and baseline measurements (ADASFA). Several potential confounding variables (covariates) were also recorded, including age, BMI, baseline ADAS1 score, and center.

The paper also utilizes simulated data to rigorously compare the performance of ESL and OLS under varying degrees of contamination (10%, 20%, and 30%) to demonstrate the robustness of the ESL method.

Key Insights and Ideas

  • Non-normality is a common problem in clinical trials: The paper highlights that the assumption of normally distributed residuals, often made in OLS regression, is frequently violated in real-world clinical data, impacting the reliability of efficacy estimates.
  • ESL offers superior robustness: The ESL method is shown to be significantly more robust than OLS when dealing with contaminated data or non-normal residuals. This means it's less affected by outliers or unusual data points.
  • Efficiency gains with ESL: Even when data is not heavily contaminated, ESL can provide more efficient estimates (lower standard deviation and MSE) compared to OLS, especially as contamination levels increase.
  • Practical applicability: The ESL method is demonstrated on a real-world MCI clinical trial dataset, showing that its results align with existing medical literature while providing more confidence due to the non-normal nature of the residuals.

Key Results

  • Simulation studies: In simulations with up to 30% contamination, the ESL method consistently outperformed OLS in terms of bias and Mean Squared Error (MSE). The standard deviations of ESL estimates were notably smaller, indicating greater precision. For instance, with 30% contamination, ESL showed significantly lower MSE values across various parameter estimates compared to OLS.
  • MCI study analysis: When applied to the MCI dataset, the ESL model provided parameter estimates that were consistent with medical literature. For example, it confirmed the significant influence of ADAS1 and certain centers on ADASFA. Notably, for the variable 'age', ESL indicated no significant effect on decreasing ADAS-cog, contrasting with OLS which suggested a negative effect, a difference the authors attribute to the non-normal residuals. The ESL result for the 'group' coefficient (treatment vs. control) was -0.141, with a confidence interval including zero, supporting the non-inferiority nature of the trial.

Limitations and Future Work

  • Tuning parameter selection: While the paper proposes a data-driven method for selecting the tuning parameter $\gamma$, the optimal choice of this parameter can still influence the balance between robustness and efficiency.
  • Computational complexity: Robust methods like ESL can be computationally more intensive than OLS, especially for very large datasets.
  • Future work: The authors suggest that this ESL method can be extended to handle more complex scenarios, such as interactions between covariates or non-linear relationships, and can be applied to other types of clinical trial data where non-normal outcomes are prevalent. Further exploration of its performance with different types of non-normality and outlier structures would also be beneficial.

Summary updated

Abstract

Randomized controlled trials (RCT) are widely used in clinical efficacy evaluation studies. Linear regression is a general method to evaluate treatment efficacy considering the existence of confounding variables. However, when residuals are not normally distributed, parameter estimation based on ordinary least squares (OLS) is inefficient. This study introduces an exponential squared loss (ESL) model to evaluate treatment effect. The proposed method provides robust estimation for non-normal data. Simulation results show that it outperforms ordinary least squares regression with contaminated data. In the mild cognitive impairment (MCI) efficacy evaluation study with traditional Chinese medicine, our method is applied to construct a linear efficacy evaluation model for the difference in Alzheimer's disease assessment scale-cognitive (ADAS-cog) scores between the final and baseline records (ADASFA), with the existence of confounding factors and non- normal residuals. The results coincide with existing medical literatures. This proposed method overcomes the limitation of confounding variables and non-normal residuals in RCT efficacy studies. It outperforms OLS on estimation efficiency in situations where the percentage of non-normal contamination reaches 30%. These advantages make it a good method for real-world clinical studies.
BibTeX
@inproceedings{li2019efficacy,
  author = {Yang Li and Zhang Zhang and Qian Feng and Danhui Yi and Fang Lü},
  title = {{An efficacy evaluation method for non-normal outcomes in randomized controlled trials}},
  booktitle = {Scientific},
  year = {2019},
  doi = {10.1038/s41598-019-47727-y},
}
Figure