TL;DR
This paper argues that infant looking time (LT) data, commonly used in developmental psychology, is log-normally distributed and should always be log-transformed before statistical analysis. By analyzing extensive datasets, the authors provide crucial recommendations for sample size calculations and the application of Bayesian statistics in infant LT research, aiming to improve the rigor and interpretability of findings.
What Problem Does This Paper Address?
Infant looking time (LT) is a cornerstone behavioral measure in developmental psychology, used to infer cognitive processes by observing how long infants attend to different stimuli. However, researchers often struggle with how to statistically analyze this data. A common issue is the right-skewed distribution of LT data across participants, which violates the assumptions of standard parametric tests like ANOVA and t-tests, especially with the small sample sizes typical in infant studies. This can lead to incorrect conclusions or missed significant findings. The paper aims to address this by providing a statistically sound framework for analyzing LT data, including recommendations for data transformation, sample size, and the use of Bayesian methods.
Methods
The authors employed a two-pronged approach to analyze LT data:
- Statistical Distribution Analysis: They investigated the distribution of LT data both within individual experiments (across participants) and across studies. This involved examining raw LT data and its log-transformed counterpart.
- Effect Size Estimation and Bayesian Analysis: They estimated the typical size of significant effects in LT studies and used these estimates to inform sample size calculations for traditional frequentist analyses. Crucially, they also explored the application of Bayesian statistical methods, which allow for more flexible hypothesis testing and can incorporate prior knowledge about effect sizes. They developed two Bayesian models: one assuming a fixed effect size and another assuming a variable effect size across experiments.
Datasets
The study utilized two distinct datasets:
- In-house Data Set: This dataset comprised LT data from 47 experiments conducted by the authors' lab, including individual participant observations (1,584 observations in total) from infants aged 6-15 months. This dataset included both successful and unsuccessful experiments.
- Literature Data Set: This dataset was a representative collection of published infant LT studies. It included group-level LT statistics from 149 experiments across 33 peer-reviewed articles, covering infants aged 2-24 months. This dataset was primarily used to assess the generalizability of findings from the in-house data.
Key Insights and Ideas
- Log-Normal Distribution is Key: The most significant finding is that LT data is consistently log-normally distributed across participants. This means that simply applying standard statistical tests to raw LT data is inappropriate and can lead to erroneous conclusions.
- Log-Transformation is Essential: The paper strongly advocates for log-transforming LT data before any parametric statistical analysis. This transformation normalizes the data, making it suitable for tests like t-tests and ANOVAs, and also helps to stabilize variance.
- Bayesian Approach Offers Advantages: The study highlights the benefits of Bayesian statistics for LT research. It allows researchers to incorporate prior knowledge about effect sizes, avoid the need to pre-specify sample sizes (allowing for sequential data collection), and provides a more nuanced interpretation of evidence, including the possibility of inferring the absence of an effect.
- Effect Sizes are Relatively Consistent: The analysis revealed that the magnitude of significant LT differences, when log-transformed, tends to be relatively consistent across studies, providing a basis for estimating effect sizes for both frequentist and Bayesian analyses.
Key Results
- Log-Normal Distribution Confirmed: Both the in-house and literature datasets showed that raw LT data is right-skewed, but log-transformed data closely approximates a normal distribution.
- Improved Statistical Power: Log-transformation of LT data revealed statistically significant effects in some experiments that would have been missed with raw data analysis.
- Sample Size Recommendations: Based on their effect size estimates, the authors provide concrete sample size recommendations for achieving adequate statistical power (e.g., 16 participants for a standard effect size with 0.75 probability).
- Bayesian Analysis Robustness: The Bayesian analyses demonstrated strong agreement with conventional statistical findings, often providing more nuanced evidence, particularly in cases of null or borderline effects.
Limitations and Future Work
- Generalizability of Datasets: While the authors used two substantial datasets, they acknowledge that these might not be perfectly representative of all possible LT studies. Selective publication (the "file-drawer problem") could potentially bias the literature dataset.
- Focus on Within-Subject Designs: The analysis primarily focused on within-subject LT designs, which are common but not the only type of LT study.
- Future Work: The authors suggest that their findings can inform the design of future LT experiments, particularly by providing a robust framework for applying Bayesian statistics. They also note that further research could explore the application of these principles to other types of infant behavioral data.
Summary updated