The meaning of significant mean group differences for biomarker discovery
Lists
Only you see this
Abstract
Author summary Currently, a striking paradox is often found in neuropsychiatric research. On the one hand, most clinicians and researchers accept that many neuropsychiatric conditions involve tremendous individual variability. On the other hand, the prevailing statistical approach is still the mean group comparison between “cases” and “controls.” Statistically significant mean group differences tell us that a given characteristic in brain, behaviour, or genes is on average different between the 2 groups. Yet, they do not delineate variability within groups. Moreover, using autism research as an example, we show that in up to 95% of abstracts, when reporting or interpreting findings, researchers omit the “on average.” This can be misleading because it evokes the impression as though the group-level difference would generalise to all individuals with that condition. Here, we used simulations to show that the latter statement is only true at very large effect sizes. We demonstrate that across different areas of autism research, mean group differences with small to large effects indicate that approximately 45% to 68% [cases] do not have an atypicality on cognitive tests or brain structure. However, we also show that across normal and nonnormal distributions, subgroups may exist despite small or nonsignificant overall effects. We propose practical approaches and steps for researchers to use mean group comparisons as the starting point for the discovery of clinically relevant subgroups.
BibTeX
@article{loth2021meaning,
author = {Eva Loth and Jumana Ahmad and Chris Chatham and Beatriz López and Ben Carter and Daisy Crawley and Bethany Oakley and Hannah Hayward and Jennifer Cooke and Antonia San José Cáceres and Danilo Bzdok and Emily J. H. Jones and Tony Charman and Christian F. Beckmann and Thomas Bourgeron and Roberto Toro and Jan K. Buitelaar and Declan G. M. Murphy and Guillaume Dumas},
title = {{The meaning of significant mean group differences for biomarker discovery}},
journal = {PLOS Computational Biology},
year = {2021},
doi = {10.1371/journal.pcbi.1009477},
}