Untranscribed Web Audio for Low Resource Speech Recognition

Andrea Carmantini, Peter Bell, Steve RenalsInterspeech 2019 · OpenReview

Abstract: Speech recognition models are highly susceptible to mismatch in the acoustic and language domains between the training and the evaluation data. For low resource languages, it is difficult to obtain transcribed speech for target domains, while untranscribed data can be collected with minimal effort. Recently, a method applying lattice-free maximum mutual information (LF-MMI) to untranscribed data has been found to be effective for semi-supervised training. However, weaker initial models and domain mismatch can result in high deletion rates for the semi-supervised model. Therefore, we propose a method to force the base model to overgenerate possible transcriptions, relying on the ability of LF-MMI to deal with uncertainty. On data from the IARPA MATERIAL programme, our new semi-supervised method outperforms the standard semi-supervised method, yielding significant gains when adapting for mismatched bandwidth and domain.

Introduction

Automatic Speech Recognition (ASR) systems are notoriously brittle when faced with "domain mismatch." A model trained on clean, narrowband (8kHz) conversational telephone speech (CTS) often fails when deployed on noisy, wideband (16kHz) web audio or broadcast news. For low-resource languages like Somali or Swahili, this problem is twofold: there is insufficient transcribed data to train robust models, and creating new transcriptions for diverse domains is prohibitively expensive.

Standard semi-supervised learning (SSL) typically uses a model's "best guess" transcript as ground truth. However, if the initial model is weak or the domain gap is too large, the model often defaults to "deletions" (skipping words), which poisons the training process. This paper addresses this "deletion trap" by biasing the generation of training targets.

Methodology: Biased Lattice Generation

The authors utilize Lattice-Free Maximum Mutual Information (LF-MMI), a discriminative objective function that can learn from a "lattice"—a graph of multiple word hypotheses—rather than a single transcript. This allows the model to learn even when it is uncertain.

To prevent the model from being too "shy" in unfamiliar acoustic environments, the researchers introduced Lattice Generation Biasing. The core mechanism is the Insertion Reward ($\ell$), which provides a bonus for every word inserted into the lattice. This forces the lattice to be more granular and contain more word hypotheses, providing a richer training signal.

The effect of varying insertion reward on relative WER change for VOA and YouTube (YTB) datasets. Negative values indicate improvement.

As shown in the figure above, an insertion reward of approximately 1.0 consistently yields the best relative improvements in Word Error Rate (WER) across different web audio sources. Additionally, Acoustic Scaling ($\kappa$) was used to adjust the weight of the acoustic model relative to the language model, making the targets more dependent on the actual audio signal rather than linguistic priors.

Experimental Setup and Results

The study focused on Somali, using ~80 hours of supervised narrowband CTS data and 600 hours of unlabeled wideband web audio (from Voice of America and YouTube). The evaluation was conducted on both narrowband CTS and wideband (WB) broadcast media.

Word Error Rate (WER) results for Somali on Conversational Telephone Speech (CTS) and Wideband (WB) evaluation sets across different model configurations.

The results demonstrate that "granularity beats certainty." By forcing more word insertions, the training lattices became noisier but more informative for the LF-MMI objective. The best performance was achieved by training an unsupervised model on the 600h wideband dataset using biased lattices, reducing the Somali WB WER from a baseline of 61.6% to 48.9%.

Conclusion

This work shows that biasing lattice generation to overgenerate hypotheses is a powerful technique for domain adaptation in low-resource settings. By preventing the model from defaulting to deletions, the system can effectively leverage untranscribed web audio to bridge the gap between narrowband telephone speech and wideband media. Future work will investigate automated methods for selecting optimal bias parameters and filtering unsupervised data.

Summary updated

Abstract

Speech recognition models are highly susceptible to mismatch in the acoustic and language domains between the training and the evaluation data. For low resource languages, it is difficult to obtain transcribed speech for target domains, while untranscribed data can be collected with minimal effort. Recently, a method applying lattice-free maximum mutual information (LF-MMI) to untranscribed data has been found to be effective for semi-supervised training. However, weaker initial models and domain mismatch can result in high deletion rates for the semi-supervised model. Therefore, we propose a method to force the base model to overgenerate possible transcriptions, relying on the ability of LF-MMI to deal with uncertainty. On data from the IARPA MATERIAL programme, our new semi-supervised method outperforms the standard semi-supervised method, yielding significant gains when adapting for mismatched bandwidth and domain.
BibTeX
@inproceedings{carmantini2019untranscribed,
  author = {Andrea Carmantini and Peter Bell and Steve Renals},
  title = {{Untranscribed Web Audio for Low Resource Speech Recognition}},
  booktitle = {Interspeech},
  year = {2019},
  url = {https://openreview.net/forum?id=xlVMoRvYy6},
}
Figure