Learning Structured Models for Phone Recognition

Slav Petrov, Adam Pauls, Dan KleinEMNLP 2007 · OpenReview

TL;DR

This paper introduces a "streamlined" approach to learning acoustic models for speech recognition by automatically inducing HMM structure through a split-merge Expectation-Maximization (EM) process. This data-driven method avoids manual structural biases and achieves state-of-the-art results in phone classification and competitive performance in phone recognition on the TIMIT dataset.

What Problem Does This Paper Address?

Traditional automatic speech recognition (ASR) systems rely on complex, manually designed structures within Hidden Markov Models (HMMs) to capture the nuances of speech. These structures include splitting phone states into beginning, middle, and end segments, modeling cross-phone coarticulation through context-dependent triphones, and using mixtures of Gaussians for emission densities. While empirically successful, this rich structure is heavily biased and may not be optimal. The authors question whether such extensive manual engineering is truly necessary and propose a more automatic, data-driven approach to learn HMM structure, analogous to recent successes in learning probabilistic context-free grammars.

Methods

The core of the paper is a novel HMM learning procedure that starts with a minimal monophone HMM (one state per phone, single Gaussian emission) and iteratively refines its structure. The refinement process involves two key operations:

  • Splitting: Each existing HMM state is split into two new substates. This is done hierarchically, with splits being initialized "close" to a reasonable maximum to avoid poor local optima. A small amount of randomness is added to break symmetry.
  • Merging: Splits that do not significantly improve data likelihood are reversed. The authors use an approximate likelihood ratio criterion for merging, which is computationally more efficient than an exact calculation.

This split-merge EM procedure is applied iteratively, allowing the model to automatically discover subphone structure and context dependencies without prior assumptions. The paper also discusses both hand-aligned and automatically-aligned cases for training data, and introduces a variational decoding approach for improved phone sequence extraction.

Datasets

The primary dataset used for training and evaluation is the TIMIT dataset. The authors mention partitioning it into standard training, development, and core test sets. They also note that the 61 phonetic labels in TIMIT were mapped down to 48 classes for training and then to 39 classes for evaluation, following standard practice.

Key Insights and Ideas

  • Data-Driven Structure Learning: The paper demonstrates that a highly simplified, automatic structure learning process can yield excellent results, challenging the necessity of extensive manual structural engineering in HMMs.
  • Emergent Phonetic Structure: The learned substructure and context dependencies naturally reflect phonetic classes and phenomena, suggesting the model effectively captures relevant acoustic information.
  • Split-Merge Efficiency: The hierarchical split-merge EM approach is shown to be effective in finding good HMM structures, balancing the need for complexity with the risk of overfitting.
  • Automatic Alignment Benefits: Allowing the model to learn phone boundaries automatically (rather than relying on manual alignments) provides significant improvements in phone recognition error rates.
  • Variational Decoding Advantage: Using a variational approximation for decoding, rather than a simple Viterbi state sequence extraction, offers a tangible reduction in phone error rates, albeit at a computational cost.

Key Results

  • Phone Recognition: The proposed model achieved a phone error rate of 26.4% on the TIMIT core test set, outperforming standard state-tied triphone HMMs (27.7%) and being competitive with other advanced methods.
  • Phone Classification: The model achieved state-of-the-art performance on phone classification, outperforming most other methods with an error rate of 21.4%.
  • Model Size vs. Error: Figure 3 shows a clear trend where increasing the number of states (through splitting) generally reduces phone recognition error, with the split-and-merge approach consistently outperforming the "split only" approach.
  • Automatic Alignment Improvement: Allowing automatic alignment improved error rates by up to 0.6% absolute.
  • Variational Decoding Gain: The variational decoder reduced error from 26.5% to 25.6% on a model with approximately 1000 states.

Limitations and Future Work

  • Scaling to Larger Systems: The authors acknowledge that it remains to be seen whether the benefits of their approach can be scaled to larger, more complex speech recognition systems.
  • Computational Cost of Variational Decoding: While beneficial, the variational decoder is about 3 times slower than standard decoding, which might limit its practical application.
  • Post Hoc Analysis: The authors caution that their analysis of learned structures is a post hoc interpretation and may be prone to seeing expected patterns.
  • Future Work: The paper implicitly suggests exploring how to further optimize the split-merge criteria, investigate different initialization strategies, and apply this approach to larger vocabulary ASR systems. The potential for this method to facilitate better word recognition rates in larger systems is also a key area for future investigation.

Summary updated

Abstract

We present a maximally streamlined approach to learning HMM-based acoustic models for automatic speech recognition. In our approach, an initial monophone HMM is iteratively refined using a split-merge EM procedure which makes no assumptions about subphone structure or context-dependent structure, and which uses only a single Gaussian per HMM state. Despite the much simplified training process, our acoustic model achieves state-of-the-art results on phone classification (where it outperforms almost all other methods) and competitive performance on phone recognition (where it outperforms standard CD triphone / subphone / GMM approaches). We also present an analysis of what is and is not learned by our system.
BibTeX
@inproceedings{petrov2007learning,
  author = {Slav Petrov and Adam Pauls and Dan Klein},
  title = {{Learning Structured Models for Phone Recognition}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2007},
  url = {https://openreview.net/forum?id=H1ZqBMzu-r},
}

Loading related content...

Figure