Estimating Mean and Variance from Non Standard Summary Statistics

20 concepts12 papers1 proposal

Meta-analysis relies on a "common currency" to aggregate findings across different studies. In most clinical and social science research, this currency is the sample mean and standard deviation. However, primary studies often report their results using a different "language"—the median, range, and interquartile range (IQR)—especially when the underlying data is skewed. This creates a data compatibility gap that can lead to the exclusion of valuable evidence or the introduction of systematic bias during evidence synthesis.

The Data Compatibility Gap

When researchers perform systematic reviews, they frequently encounter studies that do not report the mean or variance. For example, in the early months of the COVID-19 pandemic, Katzenschlager et al. et al. found that clinical data was overwhelmingly reported using medians and quartiles, creating a significant barrier to establishing a robust evidence base for triage tools [Can we predict the severe course of COVID-19 - a systematic review and meta-analysis of indicators of clinical outcome?].

To include these studies, researchers often use mathematical transformations to estimate the missing mean and standard deviation. As noted by Luo Yuxin et al. et al., these techniques allow for a more comprehensive quantitative analysis by maximizing the utility of available literature [Effects of uric acid-lowering therapy (ULT) on renal outcomes in CKD patients with asymptomatic hyperuricemia: a systematic review and meta-analysis]. However, this process is not a simple administrative step; it is a statistical estimation that carries inherent risks.

The Mechanics of Imputation

The intuition behind mean-imputation is that summary statistics like the median and range contain latent information about the distribution's center and spread. Early methods provided basic formulas for this conversion, but Wang et al. et al. introduced more refined formulas that improved the accuracy of these estimates [Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range].

A key advancement in this area was the realization that the reliability of summary statistics changes with the sample size. Luo et al. et al. developed methods that incorporate sample size into a smoothly changing weight, allowing the estimator to adapt to the amount of data available [Optimally estimating the sample mean from the sample size, median, mid-range, and/or mid-quartile range]. By weighting the components of the "five-number summary" (minimum, first quartile, median, third quartile, and maximum) based on sample size, these methods provide a more robust bridge between reporting formats.

The Normality Trap

The most significant danger in data conversion is the "normality trap." Most standard transformation formulas assume that the underlying data follows a normal distribution. However, as Sohn et al. et al. point out, there is a fundamental contradiction here: researchers typically report medians because their data is non-normal or skewed [Meta‐analysis of the difference of medians].

When normality assumptions are applied to skewed data, the resulting estimates are biased. Shi et al. et al. demonstrated that this bias can compromise the integrity of an entire meta-analysis, leading to misleading conclusions [Detecting the skewness of data from the five-number summary and its application in meta-analysis]. Furthermore, these transformations often underestimate uncertainty, resulting in confidence intervals that are too narrow and fail to cover the true population parameter.

This "statistical noise" has real-world consequences. In a meta-analysis of esmolol for sepsis, Ya et al. et al. noted that the approximation process can reduce the precision of the pooled effect size, potentially affecting the reliability of clinical findings [The role of esmolol in sepsis: a meta-analysis based on randomized controlled trials].

Toward Direct Synthesis and Automation

To address these biases, recent research has moved in two directions: direct pooling and automated extraction.

Rather than forcing medians into a mean-based framework, some researchers advocate for meta-analyzing the difference of medians directly. Sohn et al. et al. showed that direct median-pooling is significantly more accurate for non-normal distributions [Meta‐analysis of the difference of medians]. Tools like the metamedian R package, developed by Zhao et al. et al., now provide researchers with the ability to use both mean-imputation and direct median-pooling, allowing for more inclusive and robust evidence synthesis [metamedian: An R package for meta-analyzing studies reporting medians].

Finally, the bottleneck of manually extracting these complex numerical results is being addressed through machine learning. Pogrebitskiy et al. et al. explored using Large Language Models (LLMs) to automatically identify and extract these statistics from clinical trial reports [Automatically Extracting Numerical Results from Randomized Controlled Trials with Large Language Models]. While LLMs still struggle with the most complex data formats, they represent a path toward rapid, large-scale evidence synthesis that can handle the diversity of modern scientific reporting.

Key Foundational Work

Optimally estimating the sample mean from the sample size, median, mid-range and/or mid-quartile range (2016) — contributed 2 concept elements to this direction.
Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_data_conversion_bias_598e3f19,
  title = {data conversion bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/data-conversion-bias/rdr_598e3f193f484eee9cbefdce4b055ca7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_data_conversion_bias_598e3f19_v1,
  title = {data conversion bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/data-conversion-bias/rdr_598e3f193f484eee9cbefdce4b055ca7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake