Abstract
For clinical studies with continuous outcomes, when the data are potentially skewed, researchers may choose to report the whole or part of the five-number summary (the sample median, the first and third quartiles, and the minimum and maximum values) rather than the sample mean and standard deviation. In the recent literature, it is often suggested to transform the five-number summary back to the sample mean and standard deviation, which can be subsequently used in a meta-analysis. However, if a study contains skewed data, this transformation and hence the conclusions from the meta-analysis are unreliable. Therefore, we introduce a novel method for detecting the skewness of data using only the five-number summary and the sample size, and meanwhile propose a new flow chart to handle the skewed studies in a different manner. We further show by simulations that our skewness tests are able to control the type I error rates and provide good statistical power, followed by a simulated meta-analysis and a real data example that illustrate the usefulness of our new method in meta-analysis and evidence-based medicine.
Introduction: The "Normality Trap" in Meta-Analysis
In evidence-based medicine, meta-analysis serves as a cornerstone for pooling results from multiple studies to reach robust conclusions. Most standard meta-analytic methods require the sample mean and standard deviation (SD). However, clinical data is often non-normally distributed. In such cases, researchers frequently report a five-number summary—comprising the minimum, first quartile (Q1), median, third quartile (Q3), and maximum—instead of the mean and SD.
To include these studies in a meta-analysis, practitioners often use transformation formulas to estimate the mean and SD. A critical flaw in this approach is that these formulas typically assume the underlying data follows a normal (bell-shaped) distribution. When data is skewed, these transformations produce inaccurate estimates, leading to biased results and potentially misleading medical guidelines.
Methodology: Detecting Skewness from Summary Statistics
To address this "normality trap," the researchers propose three new skewness tests ($T_1, T_2, T_3$) tailored to how summary data is commonly reported:
- Scenario 1 ($S_1$): Minimum, Median, Maximum, and sample size ($n$).
- Scenario 2 ($S_2$): Q1, Median, Q3, and $n$.
- Scenario 3 ($S_3$): The full five-number summary and $n$.
The underlying logic is that in a normal distribution, the median should be roughly equidistant from the lower and upper bounds. By deriving the exact probability distributions for these differences, the authors provide a robust statistical check. Furthermore, they offer "rule of thumb" formulas for practitioners, allowing for quick skewness detection without complex software.
Validation and Performance
The proposed methods were validated using both simulated distributions and real-world clinical data.
Simulated Distributions
The power of the skewness tests was evaluated against four common non-normal distributions: Skew-normal, Half-normal, Log-normal, and a Mixture-normal distribution. These distributions represent various degrees and types of skewness encountered in clinical research.
Key Results
- Type I Error Control: Simulations confirmed that the tests maintain a consistent 5% false-positive rate across various sample sizes.
- Statistical Power: The tests demonstrate high power in correctly identifying skewness, particularly as sample size increases.
- Bias Reduction: In simulated meta-analyses, excluding studies that fail the skewness test (or using appropriate non-normal transformations) significantly outperformed the standard practice of assuming normality for all studies.
Clinical Application: COVID-19 and Liver Dysfunction
The utility of this method was demonstrated by re-analyzing a 2020 meta-analysis on COVID-19's impact on liver dysfunction (ALT levels). The original analysis found no statistically significant difference between survivors and non-survivors. However, by applying the new skewness test, the authors identified several skewed studies. Properly handling these studies revealed a significant difference in ALT levels, potentially altering clinical interpretations.
Conclusion
Detecting skewness is a vital safety check for meta-analysis. By providing a clear decision framework and easy-to-use formulas, this research helps ensure that clinical conclusions are based on reliable data transformations. An online calculator is available to assist the medical community in implementing these tests immediately.
Summary updated