Who Said What WSW 2.0? Enhanced Automated Analysis of Preschool Classroom Speech

Anchen Sun, Tiantian Feng, Gabriela Gutierrez, Juan J. Londono, Anfeng Xu, Batya Elbaum, Shrikanth Narayanan, Lynn K. Perry, Daniel S. Messinger

Introduction

Understanding teacher-child interactions in preschool settings is fundamental to early childhood education research. However, the sheer volume of audio data generated in these environments creates a massive technical bottleneck. Traditional manual transcription is labor-intensive, often requiring hours of human effort for a single minute of audio. While commercial solutions exist, they often lack transparency and struggle with the high-noise, multi-speaker dynamics of a classroom.

WSW 2.0 is an open-source automated framework designed to overcome these challenges. By integrating state-of-the-art deep learning models for speaker classification and speech transcription, it enables large-scale, longitudinal analysis of classroom discourse with high reliability.

Methodology

The WSW 2.0 framework utilizes a multi-stage automated pipeline to process raw audio into actionable linguistic data.

The WSW 2.0 automated pipeline and reliability evaluation workflow.

The system consists of two primary deep learning components:

  1. Speaker Classification: An Egocentric Speaker Classification (ESC) model based on wav2vec2 determines whether an utterance was produced by a child or a teacher.
  2. Speech Transcription: OpenAI’s Whisper model (specifically the large-v2 variant) is employed for automated speech recognition (ASR).
  3. Feature Extraction: The pipeline automatically calculates developmental indices such as Mean Length of Utterance (MLU), Lexical Diversity (LD), and conversational dynamics like question frequency and "responsivity" (the proportion of child utterances followed by a teacher response within 2.5 seconds).

Datasets and Validation

The framework was validated using a Reliability Dataset of 235 minutes of expert-coded audio. To demonstrate scalability, the researchers applied the tool to a Longitudinal Scalability Dataset comprising 1,592 hours of classroom recordings collected over two academic years. This massive dataset, containing over 1.4 million utterances, was processed in approximately two weeks—a task that would be impossible for human coders.

Key Insights

A notable finding was the performance difference between Whisper model versions. Contrary to expectations, Whisper large-v2 outperformed large-v3 for child-specific metrics, particularly child MLU. This highlights the importance of selecting models based on specific acoustic tasks rather than just version numbers.

Furthermore, the study addressed the "Child Gap" in ASR. While the system is highly effective, the Word Error Rate (WER) for teachers (.119) remains significantly lower than for children (.238), reflecting the inherent difficulty of processing child speech in noisy environments.

Results and Reliability

The WSW 2.0 framework demonstrated high reliability across various linguistic and conversational metrics when compared to expert human annotations.

Intraclass Correlation (ICC) results comparing WSW 2.0 and WSW 1.0 against expert human coding.

Key performance metrics include:

  • Classification Accuracy: An overall accuracy of 84.6% (weighted F1 of .845) in identifying speakers.
  • Transcription Reliability: Intraclass Correlation (ICC) values for features like question-asking and MLU ranged from .64 to .98, indicating strong agreement with human experts.
  • Enhanced Performance: As shown in the comparison with the previous version (WSW 1.0), the new ESC model significantly improves the reliability of child-specific metrics, such as question utterances and response proportions.

Limitations and Future Directions

Despite its strengths, WSW 2.0 faces challenges with overlapping speech (cross-talk) and the persistent performance gap in child ASR. Future work will focus on improving semantic analysis—measuring the topical alignment between teachers and children—and further refining models for the unique acoustic profiles of young learners.

As an open-source tool, WSW 2.0 provides a transparent and scalable foundation for researchers to design and evaluate educational interventions that support early language development at scale.

Summary updated

BibTeX
@inproceedings{sun2025who,
  author = {Anchen Sun and Tiantian Feng and Gabriela Gutierrez and Juan J. Londono and Anfeng Xu and Batya Elbaum and Shrikanth Narayanan and Lynn K. Perry and Daniel S. Messinger},
  title = {{Who Said What WSW 2.0? Enhanced Automated Analysis of Preschool Classroom Speech}},
  booktitle = {ICDL},
  year = {2025},
  doi = {10.1109/icdl63968.2025.11204438},
}
Figure