Lexical Access for Speech Understanding using Minimum Message Length Encoding

Ian E. Thomas, Ingrid Zukerman, Jonathan J. Oliver, David W. Albrecht, Bhavani RaskuttiUAI 1997

TL;DR

This paper proposes a novel approach to the "lexical access problem" in speech recognition, which is determining the sequence of words from a stream of phonemes. They leverage the Minimum Message Length (MML) principle, an information-theoretic framework, to model sentences using phoneme realizations, word information, and part-of-speech data. This method offers a unified way to integrate various information sources for more accurate and efficient speech understanding.

What Problem Does This Paper Address?

The core challenge addressed is the "lexical access problem" in speech recognition. After a low-level phoneme recognizer outputs a sequence of basic speech sounds, the system needs to figure out which words these phonemes correspond to. This is tricky because:

  • Phoneme Recognition Errors: The input phoneme sequence might not perfectly match the canonical phonemes of words due to mis-recognition (equipment issues, poor recordings) or mis-pronunciation (accents, dialects). This leads to insertions, deletions, and substitutions of phonemes.
  • Continuous Speech Ambiguity: In continuous speech, word boundaries are not clearly defined. Co-articulation (sounds blending together) further blurs these boundaries, making it hard to segment the phoneme stream into individual words.
  • Sentence-Level Uncertainty: Beyond individual words, the system doesn't know the total number of words in a sentence or their start/end points.

Traditional approaches often involve complex graph searches or hypothesize-and-test paradigms. This paper aims to provide a more unified and information-theoretic framework to tackle these challenges.

Methods

The paper's central contribution is the application of the Minimum Message Length (MML) criterion to lexical access. MML is an information-theoretic principle that seeks the shortest possible message to describe data, which in this context means finding the sentence that best explains the input phoneme sequence.

The MML approach models a sentence as a two-part message:

  1. Model Description (Language Model): This part encodes the hypothesized sentence itself. It uses an N-gram model (specifically, trigrams for part-of-speech tags and bigrams for words) to estimate the probability of word sequences. This favors sensible English sentences by considering word order and syntactic roles (part-of-speech).
  2. Object Description (Phoneme Realization Difference): This part describes how well the hypothesized sentence's phonemes match the actual input phonemes. It quantifies the "deviation" or "error" using an edit distance algorithm to find the closest phoneme realization for each word.

The core idea is that the "best" sentence is the one that minimizes the total message length, which is the sum of the lengths of these two parts. The paper also introduces a heuristic using equivalence classes of similar sounding words to speed up the search process by creating a "short-list" of likely candidates.

Datasets

  • TIMIT Corpus: This is the primary dataset used for training and evaluation. It's a collection of American-English read sentences with time-aligned acoustic-phonetic and orthographic transcriptions. The dataset contains 3696 sentences from 462 speakers across 8 dialect divisions.
  • Classic Texts: To overcome the limitations of TIMIT for language modeling (due to its small, diverse sentence set), the authors used a larger body of classic texts (e.g., Virgil's Aeneid, Emily Brontë's Wuthering Heights) to extract more robust word and part-of-speech N-grams.

Key Insights and Ideas

  • Unified Information-Theoretic Framework: MML provides a principled way to combine information from different sources (language model, lexicon, phoneme realization) into a single objective function (message length). This avoids ad-hoc weighting schemes.
  • Modeling Phoneme Deviations: Quantifying the "difference" between hypothesized phonemes and actual input phonemes using edit distance and phoneme realization frequencies is crucial for robust lexical access, especially with noisy input.
  • Balancing Language Model and Acoustic Evidence: The MML framework naturally balances the "sensibility" of a sentence (language model) with how well its phonemes match the input (phoneme realization). A sentence might be linguistically plausible but acoustically poor, or vice-versa.
  • Speeding Up Search with Equivalence Classes: Grouping similar-sounding words into equivalence classes significantly speeds up the search by reducing the number of candidates to evaluate at each step, with minimal impact on accuracy.

Key Results

  • The paper demonstrates results on a test set of sentences, showing a breakdown of message length into part-of-speech, word, and phoneme difference costs.
  • Table 1 shows an example where the "best" sentence found by the system was not the "correct" sentence, indicating that the search might miss optimal solutions or prune them too early. This highlights the trade-off between search efficiency and finding the absolute best solution.
  • Table 3 presents average word error rates for different levels of distortion. Incorporating part-of-speech information improved recognition accuracy compared to using only word bigrams.
  • Figure 2 shows a correlation between bits per phoneme and word error rate, suggesting that a higher message length per phoneme indicates a higher error rate, which could be used to assess the confidence of a recognition result.
  • The average word error rate achieved was 29%.

Limitations and Future Work

  • Language Model Data: The TIMIT corpus, while good for phonetics, is limited for language modeling. The use of external classic texts helped, but a more comprehensive language model would still be beneficial.
  • Search Optimization: The current search uses an "optimistic selection" principle and a beam threshold. The authors suggest further investigation into exploiting this principle more deeply during word evaluation to improve search efficiency.
  • Phoneme Confidence: The input is assumed to be a sequence of phoneme candidates. In a realistic system, input would be a lattice of phoneme candidates with associated probabilities. Incorporating this confidence information could further improve results.
  • Other Features: The authors mention that features like phoneme duration could be incorporated as additional sources of information.
  • Co-articulation Modeling: More advanced methods for modeling co-articulation between words could enhance accuracy.

Summary updated

BibTeX
@inproceedings{thomas1997lexical,
  author = {Ian E. Thomas and Ingrid Zukerman and Jonathan J. Oliver and David W. Albrecht and Bhavani Raskutti},
  title = {{Lexical Access for Speech Understanding using Minimum Message Length Encoding}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {1997},
}
Figure