TL;DR
This paper validates the use of moderated online data collection methods for developmental research, demonstrating that results from online studies using video conferencing are comparable to traditional in-person studies. This opens doors for more accessible, diverse, and large-scale developmental research.
What Problem Does This Paper Address?
The COVID-19 pandemic accelerated the adoption of online data collection in developmental psychology. However, many researchers remain hesitant due to uncertainties about the validity of these methods, particularly for young children and infants. Specifically, there's a lack of clear guidance on how to effectively design and implement online studies, and limited direct comparisons between in-person and online data collection for the same tasks. This paper aims to bridge this gap by providing practical advice and empirical evidence for the reliability of moderated online data collection.
Methods
The authors propose and evaluate "moderated online data collection," which involves real-time, face-to-face interactions with participants via video conferencing software (e.g., Zoom). This contrasts with "unmoderated" online methods where participants complete tasks independently. The paper outlines best practices for designing these moderated sessions, focusing on maintaining participant engagement and addressing potential technical issues.
To validate these methods, the authors present four studies that were conducted both in-person and via moderated online sessions. These studies employed different developmental measures:
- Choice and Verbal Measures (Studies 1 & 2): These studies involved children aged 4-9 making choices or providing verbal responses, adapted for online by using visual cues and verbal prompts.
- Looking Time Measures (Studies 3 & 4): These studies focused on infants, using preferential looking and violation-of-expectation paradigms, adapted for online by presenting stimuli via screen-sharing and measuring gaze duration.
A key methodological contribution is the direct comparison of results from the in-person and online versions of each study, followed by a meta-analysis to aggregate findings across all four studies.
Datasets
Not explicitly discussed as distinct datasets. The paper presents data from four newly conducted studies (Studies 1-4) that were run in parallel in-person and online. The data from Studies 1, 2, and 4 are available via OSF, and data for Study 3 is referenced as being in Woo and Spelke (2020).
Key Insights and Ideas
- Moderated online studies are a viable alternative: The core finding is that, for the measures tested, moderated online data collection yields results comparable to traditional in-person methods. This is a significant finding for the field.
- Adaptability of measures: The paper demonstrates that common developmental measures, including choice tasks, verbal responses, and infant looking-time paradigms, can be successfully adapted for moderated online settings.
- Importance of careful design: While online methods offer flexibility, careful consideration of factors like software choice, participant engagement strategies (e.g., keeping sessions short, using social cues), and robust handling of technical issues is crucial for successful implementation.
- Potential for increased accessibility and diversity: Online methods have the potential to overcome geographical barriers, allowing researchers to recruit from more diverse populations and conduct larger-scale studies, which is vital for the generalizability of findings.
Key Results
The four studies presented showed consistent patterns between in-person and online data collection:
- Study 1 (4-5 year olds, competence inference): Children's performance in inferring agent competence based on task difficulty was comparable across in-person (90% correct) and online (85% correct) conditions.
- Study 2 (6-9 year olds, social cues): Children's ability to use adult surprise to infer competence was also similar, with online results mirroring those of the older age group in the in-person study.
- Study 3 (15-month-olds, social evaluations): Infants' preferential reaching (in-person) and looking (online) patterns regarding agents' intentions were comparable.
- Study 4 (6-month-olds, cost minimization): Infants' looking times at efficient versus inefficient actions were similar across in-person and online conditions.
The meta-analysis of these studies estimated a small, non-significant negative effect of online data collection (p = 0.58), indicating no substantial difference in effect sizes between the two methods.
Limitations and Future Work
The authors acknowledge several limitations:
- Focus on social cognition: The studies primarily involved animated agents, which may inherently be more engaging for children. The generalizability of these findings to studies involving inanimate objects, sounds, or more abstract stimuli remains an open question.
- Limited range of measures: The paper focused on choice, verbal, and looking-time measures. The efficacy of other, more continuous measures (e.g., rating scales, free-form responses) in moderated online settings needs further investigation.
- Participant demographics: The studies were conducted in the US with participants likely having reliable internet access and familiarity with technology. The applicability to populations with limited internet access or lower digital literacy is not yet established.
Future work should explore a broader range of measures and research questions, investigate the efficacy of online methods in diverse populations, and continue to develop innovative strategies for online data collection. The authors express optimism that online methods will continue to improve the accessibility, diversity, and replicability of developmental science.
Summary updated