Acoustic modeling with bootstrap and restructuring for low-resourced languages

Xiaodong Cui, Jian Xue, Pierre L. Dognin, Upendra V. Chaudhari, Bowen ZhouInterspeech 2010 · OpenReview

Abstract

This paper investigates an acoustic modeling approach for low-resourced languages based on bootstrap and model restructuring. The approach first creates an acoustic model with redundancy by averaging over bootstrapped models from resampled subsets of sparse training data, which is followed by model restructuring to scale down the model to a desired cardinality. A variety of techniques for Gaussian clustering and model refinement are discussed for the model restructuring. LVCSR experiments are carried out on Pashto language with up to 105 hours of training data. The proposed approach is shown to yield more robust acoustic models given sparse training data and obtain superior performance over the traditional training procedure.

Introduction

Building Automatic Speech Recognition (ASR) systems typically requires massive amounts of transcribed audio. For "low-resourced" languages—such as Pashto, Farsi, or Dari—high-quality data is scarce and expensive to produce. When training data is sparse, standard Hidden Markov Models (HMMs) often suffer from poor parameter estimation, leading to high Word Error Rates (WER).

Researchers at IBM have developed a two-stage "bootstrap and restructuring" pipeline to address this. While ensemble methods like bagging can improve robustness by averaging multiple models, they often result in models that are too large for real-world deployment on mobile devices. This research focuses on gaining the robustness of an ensemble while maintaining the efficiency of a single, compact model.

Methodology

The proposed framework consists of two primary stages designed to maximize the utility of limited data while ensuring computational efficiency.

1. Bootstrap and Averaging

In the first stage, the sparse training data is resampled into multiple subsets, each covering approximately 70% of the total data. An HMM is trained on each subset, and these models are subsequently averaged to create a single "mega-model." This process effectively "smooths out" the noise inherent in small datasets, providing a more stable estimate of acoustic properties. However, the resulting model contains a massive number of Gaussian components, making it too heavy for practical use.

2. Model Restructuring

To make the model usable, it is scaled down through two sub-steps:

  • Gaussian Clustering: A greedy, bottom-up approach merges the most similar Gaussians until a target size is reached. The researchers tested several mathematical criteria for similarity, including KL Divergence, Entropy, and Bayes Error.
  • Model Refinement: After clustering, parameters are tuned to minimize the distance between the new compact model and the original ensemble. Advanced techniques like Variational EM and Monte Carlo-based KL minimization were explored.
The proposed acoustic modeling pipeline consisting of bootstrap and restructuring stages.

Experimental Setup

The study utilized Pashto language data collected under the DARPA Transtac project. To evaluate scalability, three training set sizes were tested: 35 hours, 60 hours, and 105 hours of audio. Evaluation was performed on a 10-hour held-out dataset using a dictionary of 30,000 words and a trigram language model with 1.2 million n-grams.

Key Results and Discussion

The proposed method consistently outperformed the standard training baseline across all data sizes. Among the clustering criteria, the Entropy criterion performed slightly better than KL divergence or Bayes error.

As shown in the results table, the refinement process is crucial. Using Monte Carlo methods to refine the HMM on actual observation sequences (the mchmm approach) yielded the best final performance. For instance, in the 35-hour set, the bootstrap/restructuring method improved the WER from 47.7% to 45.8% (a 1.9% absolute gain) for Maximum Likelihood models.

WER results for various clustering and refinement techniques across different training data sizes.

When combined with advanced discriminative training (fMMI+BMMI), the approach reached a WER of 34.9% on the 105-hour set, representing a significant improvement over the previous state-of-the-art baseline. Crucially, the restructuring successfully compressed models with over 1 million Gaussians down to 60,000–80,000 Gaussians while retaining the accuracy gains of the larger ensemble.

Conclusion

The "bootstrap and restructuring" pipeline provides a robust solution for acoustic modeling in low-resource scenarios. By intentionally introducing redundancy and then mathematically compressing it, the system supports deeper decision trees and more stable estimates than traditional training. While the process adds computational overhead during the one-time training phase, it delivers a highly efficient and accurate model for deployment. Future work may explore the application of these techniques to other low-resource languages and their interaction with dynamic decoding graphs.

Summary updated

Abstract

This paper investigates an acoustic modeling approach for low-resourced languages based on bootstrap and model restructuring. The approach first creates an acoustic model with redundancy by averaging over bootstrapped models from resampled subsets of sparse training data, which is followed by model restructuring to scale down the model to a desired cardinality. A variety of techniques for Gaussian clustering and model refinement are discussed for the model restructuring. LVCSR experiments are carried out on Pashto language with up to 105 hours of training data. The proposed approach is shown to yield more robust acoustic models given sparse training data and obtain superior performance over the traditional training procedure.
BibTeX
@inproceedings{cui2010acoustic,
  author = {Xiaodong Cui and Jian Xue and Pierre L. Dognin and Upendra V. Chaudhari and Bowen Zhou},
  title = {{Acoustic modeling with bootstrap and restructuring for low-resourced languages}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=eCnut84TWb},
}
Figure