Generating Synthetic Children’s Acoustic Models from Adult Models

Andreas Hagen, Bryan Pellom, Kadri HaciogluNAACL 2009

TL;DR

This paper introduces a novel method to create acoustic models for children's speech recognition using only adult speech data. By transforming adult male models into synthetic children's models, they achieve significant improvements in pronunciation verification tasks, especially when real children's data is scarce.

What Problem Does This Paper Address?

Language learning is increasingly important, and effective speech recognition tools are crucial for educational applications, particularly for children. However, collecting sufficient children's speech data is challenging and costly, hindering the development of accurate speech recognition systems for this demographic. Children's speech has distinct characteristics, such as higher pitch and formants, and greater variability compared to adult speech. This paper aims to bridge the gap by generating synthetic children's acoustic models from readily available adult data, thereby reducing the reliance on expensive and difficult-to-obtain children's speech datasets.

Methods

The core of the proposed method involves creating a synthetic children's acoustic model by transforming an adult female model. The process begins by training separate male and female acoustic models from adult speech data. A linear transformation matrix is then estimated to map the mean vectors of the male HMM states to the closest corresponding mean vectors in the female model. This transformation captures the differences between male and female speech.

Crucially, this transformation matrix is then adjusted by raising it to a power, p. This power-adjusted transformation is applied to the female acoustic model to generate the synthetic children's model. The optimal value of p is determined empirically, either by evaluating on available children's test data or by estimating it from a language where sufficient data for all three groups (male, female, and children) exists, based on the assumption that p is largely language-independent. This approach leverages the known acoustic differences between male, female, and children's speech to synthesize a model suitable for children.

Datasets

The experiments were conducted using two languages: US English and Spanish. For both languages, the authors had access to over 20 hours of adult male, adult female, and children's speech data to train acoustic models.

For evaluation, specific test corpora were used:

  • US English: 2,182 utterances from 22 native speakers aged 5-14.
  • Spanish: 2,598 utterances from 19 speakers aged 8-13.

The paper does not introduce new benchmark datasets but rather utilizes existing corpora for evaluation.

Key Insights and Ideas

  • Leveraging Adult Data: The most significant insight is that adult speech data, particularly the relationship between male and female models, can be effectively used to synthesize children's speech models, bypassing the need for extensive children's data.
  • Transformation Power (p): The introduction of the power parameter p is a key innovation. It allows for fine-tuning the transformation, enabling the synthetic model to better match the specific acoustic characteristics of children's speech, which are more pronounced than just the difference between male and female speech.
  • Language Independence of p: The hypothesis that the optimal power p is relatively language-independent is a strong claim. If proven true, it significantly simplifies the process of creating synthetic models for new languages.
  • Synthetic Model Performance: The synthetic models, while not outperforming a real children's model, achieve performance levels that are significantly better than the adult female model and competitive with or superior to existing methods.

Key Results

The paper reports results in terms of Equal Error Rate (EER) for utterance verification tasks.

  • US English: The optimal power p was found to be around 0.25. The synthetic model achieved an EER of 2.36%, which is a 19% relative improvement over the adult male model (4.07%) and a 3.7% relative improvement over the adult female model (2.92%). It also outperformed the available real children's data-based model by 13% relative.
  • Spanish: The optimal power p was found to be around 0.27. The synthetic model achieved an EER of 2.09%, outperforming the adult male model (5.62%) and the adult female model (2.17%). Interestingly, the synthetic model also outperformed the real children's model (2.40%) in this case.

Limitations and Future Work

  • Real Children's Model Performance: The authors acknowledge that the gold standard remains a model trained directly on children's data, which consistently yields the best EER performance. The synthetic model is presented as a strong alternative when such data is unavailable.
  • Explanation for Spanish Children's Model: The paper notes that in the Spanish evaluation, the real children's model performed worse than both the female and synthetic models. They hypothesize this could be due to diverse and variable training data for the children's model, which might hurt classification performance.
  • Comparison with Adaptation Techniques: The authors suggest future work to compare their method with established adaptation techniques like MLLR, especially when limited amounts of children's data are available. They are interested in whether their technique remains superior or if speaker adaptation becomes more competitive with increasing children's data.
  • Further Investigation of p: While the paper suggests p is language-independent, further research is needed to definitively prove this claim across a wider range of languages and acoustic conditions.

Summary updated

BibTeX
@inproceedings{hagen2009generating,
  author = {Andreas Hagen and Bryan Pellom and Kadri Hacioglu},
  title = {{Generating Synthetic Children’s Acoustic Models from Adult Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
}
Figure