Enhancing children's speech recognition under mismatched condition by explicit acoustic normalization

Shweta Ghai, Rohit SinhaInterspeech 2010

TL;DR

This paper tackles the significant challenge of recognizing children's speech, which is acoustically very different from adult speech, by proposing explicit normalization of pitch and speaking rate. Their approach, when combined with standard model adaptation techniques, significantly improves recognition accuracy, outperforming existing methods.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems are typically trained on adult speech. When these systems are used to recognize children's speech, performance degrades dramatically due to a large acoustic and linguistic mismatch. This mismatch arises from differences in children's vocal tract length, formant frequencies, pitch, average phone duration, and speaking rate. While existing techniques like Vocal Tract Length Normalization (VTLN) and Maximum Likelihood Linear Regression (MLLR) aim to bridge this gap, they often rely on linear transformations that may not fully capture the complex acoustic variations in children's speech. This paper investigates whether explicitly normalizing key acoustic features like pitch and speaking rate, in addition to standard adaptation methods, can further enhance children's speech recognition accuracy.

Methods

The authors explore the effectiveness of explicit acoustic normalization of pitch and speaking rate, combined with established model adaptation techniques. The standard techniques reviewed include:

  • MLLR (Maximum Likelihood Linear Regression): Adapts the mean and/or variance parameters of Gaussian distributions in Hidden Markov Models (HMMs) using linear transformations. They consider 'MLLR-MEAN' (adapting means) and 'MLLR-COV' (adapting variances).
  • CMLLR (Constrained MLLR): Adapts features by applying linear transformations to the observation vectors.
  • VTLN (Vocal Tract Length Normalization): Warps the frequency axis of the speech spectrum to reduce inter-speaker variability related to vocal tract length.

The novel contribution lies in the explicit normalization of pitch and speaking rate. For pitch normalization, the average pitch of each test signal is transformed to seven different values. For speaking rate normalization, the duration of each test signal is scaled by factors ranging from 0.5 to 1.0. These normalization steps are performed using the pitch-synchronous time scaling (PSTS) method. The optimal normalization parameters for pitch and speaking rate are estimated using a Maximum Likelihood (ML) grid search, similar to VTLN. These explicit normalization techniques are then applied in conjunction with MLLR and CMLLR.

Datasets

The paper uses the following datasets:

  • TIDIGITS: Used for the connected digit recognition task. Both adult (training) and children (testing) speech data are utilized.
  • Wall Street Journal (WSJO): Used for adult speech data in the continuous speech recognition task.
  • PFSTAR British English Corpus: Used for children's speech data in the continuous speech recognition task.

Key Insights and Ideas

  • Linear Transformations Aren't Enough: The core idea is that while linear adaptation methods (MLLR, CMLLR) help, they might not fully address the non-linear acoustic variations present in children's speech. Explicitly targeting features like pitch and speaking rate offers a more direct way to compensate for these specific mismatches.
  • Pitch Normalization's Dual Effect: Pitch normalization seems to reduce the variance of features. This is beneficial for methods like CMLLR and MLLR-MEAN, leading to better adaptation of means. However, for MLLR-COV, it slightly reduces the relative improvement, possibly because there's less variance to adapt in the first place.
  • Synergy Between Explicit Normalization and Adaptation: The most significant gains are achieved when explicit pitch and speaking rate normalization are combined with model adaptation techniques like CMLLR. This suggests a powerful synergy where explicit feature correction primes the data for more effective model-based adaptation.

Key Results

The paper reports significant improvements in Word Error Rate (WER) for children's speech recognition on adult-trained models:

  • Connected Digit Recognition:
    • Explicit acoustic normalization (pitch and speaking rate) achieved a 13% relative improvement over the combined VTLN and CMLLR performance.
  • Without VTLN, explicit pitch and speaking rate normalization combined with CMLLR yielded improvements of around 47-53% over the baseline for original children's speech.
  • Continuous Speech Recognition:
    • Explicit acoustic normalization (pitch and speaking rate) achieved a 5% relative improvement over the combined VTLN and CMLLR performance.
  • For original children's speech, MLLR-MEAN, MLLR-COV, and CMLLR provided relative improvements of 10%, 9.3%, and 12.7% over the baseline, respectively. Explicit normalization further boosted these gains.

Limitations and Future Work

  • Limitations: The paper notes that while explicit normalization helps, the transformations are not constrained to be linear, which is a strength. However, the specific range and step sizes for pitch and speaking rate normalization were chosen based on adult speech distributions, implying a potential for further optimization. The study also acknowledges that the reduction in variances after pitch normalization might affect MLLR-COV performance.
  • Future Work: The paper implicitly suggests exploring other explicit normalization techniques for different acoustic mismatch sources (e.g., formant frequencies) and investigating more advanced non-linear transformation models. Further research could also focus on optimizing the normalization parameters and exploring adaptive normalization strategies that are more dynamic and data-driven.

Summary updated

BibTeX
@inproceedings{ghai2010enhancing,
  author = {Shweta Ghai and Rohit Sinha},
  title = {{Enhancing children's speech recognition under mismatched condition by explicit acoustic normalization}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure