Prosody Usage Optimization for Children Speech Recognition with Zero Resource Children Speech

Chenda Li, Yanmin QianInterspeech 2019 · OpenReview

Abstract

Children’s speech recognition remains a significant challenge for automatic speech recognition (ASR). Due to the higher cost of data collection, most current ASR systems are optimized using adult speech with limited or no children’s speech data. The resulting acoustic mismatch is the primary cause of performance degradation. We propose several approaches to improve children’s speech recognition without using any children’s speech data. By developing a better utilization strategy for prosody-based features, including pitch modification and a novel joint-decoding architecture, we achieve a relative ~20% Word Error Rate (WER) reduction on children's speech while maintaining performance on adult speech.

Introduction

Automatic Speech Recognition (ASR) systems often fail to accurately transcribe children's voices. This is primarily due to the physical differences in vocal tracts: children have shorter vocal tracts, leading to higher fundamental frequencies (pitch) and distinct resonance patterns compared to adults. Because most large-scale datasets are composed of adult speech, models suffer from a severe acoustic mismatch. Collecting children's speech is difficult due to privacy and logistical hurdles. This work explores "Zero Resource" techniques to bridge this gap by transforming adult speech to mimic children's prosody.

Methodology

The proposed system optimizes prosody usage through three main components: pitch-aware feature integration, prosody modification of training data, and a joint-decoding framework.

Prosody Modification

To simulate children's speech, adult audio is processed using resampling and Waveform Similarity Overlap-Add (WSOLA). Resampling shifts frequency components upward to emulate a smaller vocal tract, while WSOLA restores the original duration, ensuring the pitch changes without altering the speaking rate. This allows the model to learn from "child-like" acoustic data derived entirely from adult recordings.

Comparison of spectrograms: (a) original adult speech and (b) prosody-modified speech simulating children's acoustic characteristics.

Joint Decoding Architecture

A key challenge in modifying speech is the potential for performance degradation on the original (adult) domain. To mitigate this, we implemented a joint-decoding strategy. As shown in the architecture below, both the original wave and the prosody-modified wave are processed. Features are extracted from both streams and fed into a shared DNN acoustic model. The system then combines the likelihoods from both streams to produce a joint probability for the decoding pipeline, allowing the model to remain robust across both adult and child acoustic spaces.

Proposed joint-decoding architecture incorporating original and prosody-modified speech streams.

Experimental Results

The methods were evaluated on a 400-hour Mandarin adult speech corpus. Mandarin is a tonal language, making pitch features particularly relevant for both meaning and speaker characterization.

System Configuration Adult WER (%) Child WER (%)
Baseline 16.26 29.23
+ Pitch Features 15.65 28.66
Full Optimization (Joint Decoding) 15.79 23.71

The results demonstrate that adding explicit pitch features (probability of voicing, pitch, and delta-pitch) provides an initial boost. However, the full optimization—combining modified training data and joint decoding—yielded a significant 20% relative reduction in WER for children's speech (from 29.23% to 23.71%). Crucially, adult speech performance remained stable (15.79% vs 16.26% baseline), confirming the robustness of the joint-decoding approach.

Conclusion

This research proves that signal processing techniques can effectively compensate for the lack of children's training data in ASR. By mathematically shifting adult pitch and employing a dual-stream decoding architecture, we can significantly improve accessibility for younger users without sacrificing general model accuracy. Future work will explore the computational efficiency of these multi-stream models for deployment on edge devices like smart toys.

Summary updated

Abstract

Children’s speech recognition remains a big challenge for automatic speech recognition. Due to the more difficult process and higher cost on data collection, most current ASR systems are optimized only using lots of adult speech with limited or even none children’s speech. Accordingly, the acoustic mismatch between children’s and adult speech is the primary reason for the ASR performance degradation when facing children’s speech. To overcome this problem, we proposed several approaches to improve children’s speech recognition without using any children’s speech data. A better utilization strategy on prosody-based features is developed. First, pitch and prosody modification is explored in both training and testing respectively, which can significantly reduce the mismatch between two types of speech. Furthermore, joint-decoding with both the prosody modified speech and the original speech is designed to get a more robust performance on both children’s and adult speech. Experiments are evaluated on a Mandarin speech recognition task, with only 400-hour adult speech in the training. The results show that our proposed method can obtain a large gain on children’s speech, with relative ~20% WER reduction compared to the baseline, and also no obvious degradation is observed on the adult speech for the proposed system.
BibTeX
@inproceedings{li2019prosody,
  author = {Chenda Li and Yanmin Qian},
  title = {{Prosody Usage Optimization for Children Speech Recognition with Zero Resource Children Speech}},
  booktitle = {Interspeech},
  year = {2019},
  url = {https://openreview.net/forum?id=ouXhF82hBj},
}
Figure