TL;DR
This paper investigates the acoustic differences between spontaneous and read speech in children aged 8-13, finding that spontaneous speech is shorter and acoustically less distinct. Crucially, automatic speech recognition (ASR) systems perform significantly worse on spontaneous speech, highlighting the challenge of modeling disfluencies and natural speech variability beyond simple acoustic differences.
What Problem Does This Paper Address?
While automatic speech recognition (ASR) holds immense potential for children (e.g., in educational tools), recognizing children's speech is inherently more challenging than adult speech due to increased variability. Previous research has focused on improving recognition for read children's speech. However, this paper tackles the more difficult problem of spontaneous children's speech. Spontaneous speech, characterized by disfluencies (hesitations, false starts, fillers) and other natural speech phenomena, is not only linguistically but also acoustically more variable than read speech. Modeling these spontaneous speech phenomena is an open challenge, and their impact on ASR performance for children is not well understood. The authors aim to bridge this gap by analyzing acoustic differences and evaluating ASR performance on spontaneous vs. read children's speech.
Methods
The authors employed two main approaches: acoustic analysis and ASR experiments.
Acoustic Analysis: * Phone Duration: They analyzed mean phone duration across different ages, comparing read and spontaneous speech. This involved automatically segmenting speech using forced alignment with Hidden Markov Models (HMMs). * Acoustic Space Characterization: To quantify the distinctiveness of speech sounds, they measured the "scattering" of acoustic features. This was done by modeling each phone with a Gaussian distribution and calculating the average Bhattacharyya distance between these distributions. A larger distance implies better discrimination between phones. They also calculated an "acoustic-space reduction ratio" to quantify how much the acoustic space shrinks when moving from read to spontaneous speech.
ASR Experiments: * Acoustic Modeling: They used state-tied, cross-word triphone HMMs with Phonetic Decision Trees (PDTs) for tying states. Gaussian Mixture Models (GMMs) with up to 8 components were used for output distributions. * Feature Extraction: Speech frames were parameterized using 13 Mel-Frequency Cepstral Coefficients (MFCCs) along with their first and second-order time derivatives. Cepstral mean subtraction was applied. * Language Modeling: A unigram language model was used. * Speaker Adaptation: They investigated several speaker adaptive techniques: Vocal Tract Length Normalization (VTLN), Speaker Adaptive Training (SAT), and Constrained MLLR (CMLSN). These methods were applied in a "2-pass" decoding scenario where the first pass output was used to adapt the models for the second pass.
Datasets
- CID Corpus: A read speech database of American English collected from children aged 5-18 and adults. For this study, a subset of 6 speakers per age (8-13) was used, with two repetitions of five sentences each.
- CHIMP Corpus: An American English spontaneous speech database collected from children aged 8-14 during a Wizard of Oz experiment playing a computer game. Audio was recorded using head-mounted and desktop microphones. For this work, data from 144 children using the head-mounted microphone was used. This corpus was manually annotated for spontaneous phenomena like hesitations and filled pauses.
The CHIMP corpus was partitioned into:
- Training Set: Used for training HMMs.
- Test Set: Further divided into:
- T1: Utterances without explicitly annotated spontaneous speech phenomena.
- T2: Utterances with explicitly annotated spontaneous speech phenomena.
- T1+T2: The combined test set.
Key Insights and Ideas
- Spontaneous Speech is Shorter: Children's spontaneous speech exhibits significantly shorter phone durations compared to their read speech, and this effect is more pronounced in older children.
- Acoustic Space Shrinks in Spontaneous Speech: The acoustic space occupied by phones in spontaneous speech is less scattered (smaller Bhattacharyya distance) than in read speech. This suggests that phones are acoustically less distinct and harder to differentiate in spontaneous speech.
- Disfluencies are a Major ASR Hurdle: The presence of explicitly annotated spontaneous speech phenomena (T2 subset) leads to a substantial degradation in ASR performance compared to speech without these phenomena (T1 subset). This indicates that current ASR systems struggle with the inherent variability introduced by disfluencies, even beyond simple acoustic differences.
- Speaker Adaptation Has Limited Impact on Spontaneous Speech: While speaker adaptive techniques (VTLN, SAT, CMLSN) improve ASR performance, they show limited effectiveness in reducing the performance gap between spontaneous and non-spontaneous speech. This suggests that the challenges posed by spontaneous speech phenomena are not solely due to inter-speaker variability but stem from other linguistic and acoustic factors.
Key Results
- Phone Duration: Mean phone duration for 13-year-olds in read speech was ~20% shorter than for 8-year-olds, while for spontaneous speech, the difference was only ~10%.
- Acoustic Space: The average Bhattacharyya distance was lower for HMMs trained on spontaneous speech (T2) compared to read speech (CID), indicating less scattered phone distributions. The acoustic-space reduction ratio was generally less than 1 for most phones when comparing T2 to T1, meaning the acoustic space shrunk.
- ASR Performance (Word Error Rate - WER):
- Baseline system on T1 (no spontaneous phenomena): 30.1% WER
- Baseline system on T2 (with spontaneous phenomena): 48.0% WER
- This represents a ~60% increase in WER for spontaneous speech.
- Speaker adaptive techniques (VTLN, SAT, CMLSN) reduced the overall WER but did not significantly close the performance gap between T1 and T2. For example, CMLSN achieved 25.0% WER on T1 and 43.3% WER on T2.
- Correlation with Spontaneous Effects: The number of hesitations/filled pauses showed the strongest positive correlation with WER (+0.65), followed by the percentage of sentences with spontaneous annotations (+0.60).
Limitations and Future Work
- Limited Impact of Speaker Adaptation: The authors acknowledge that speaker adaptive techniques showed limited effectiveness on the CHIMP test set. They hypothesize this is because the high error rate is due to linguistic/acoustic factors beyond inter-speaker variability.
- Future Work: The authors suggest that robust ASR solutions for spontaneous children's speech need to consider models beyond conventional acoustic adaptation and normalization. This implies a need for better modeling of disfluencies and other spontaneous speech phenomena directly. The paper also notes that the acoustic-space reduction ratio analysis was limited to vowel sounds and that the CID subset's results might be affected by reading ability.
Summary updated