Using prominence and phrasing predictions to improve weighted dictionary pronunciation models

Andrew RosenbergInterspeech 2012

TL;DR

This paper shows that predicting where words are emphasized (prominence) and where sentences naturally break (phrasing) can significantly improve how we model word pronunciations. By incorporating these prosodic cues, the models become more accurate and better at handling the natural variations in how people speak.

What Problem Does This Paper Address?

When we speak, our words don't always sound exactly like they do in a dictionary. This variation is influenced by many factors, including how stressed a word is and where it falls within a sentence's natural rhythm. Traditional pronunciation models often rely on a single, canonical pronunciation for each word, which can lead to errors in speech recognition systems. This paper tackles the challenge of modeling this pronunciation variability by exploring how prosodic features, specifically pitch accent (prominence) and intonational phrase boundaries (phrasing), can be used to create more flexible and accurate pronunciation models.

Methods

The authors propose using a weighted dictionary pronunciation model, which assigns probabilities to different possible pronunciations of a word. They then enhance this model by incorporating predicted prosodic events. Specifically, they use a tool called AuToBI to hypothesize the locations of pitch accents and intonational phrase boundaries for each word in their dataset. These hypothesized prosodic events are then used to condition the pronunciation model, effectively creating a model that predicts pronunciations based on both the word itself and its prosodic context. To evaluate their approach, they measure the perplexity of their models, a common metric for assessing how well a model predicts a sequence of events (in this case, pronunciations).

Datasets

The primary dataset used in this research is the Buckeye Corpus. This corpus is notable for containing autobiographical interviews that have been manually transcribed both orthographically and phonemically. Crucially, it provides both the canonical (dictionary) pronunciation and the realized pronunciation for each word, making it ideal for studying pronunciation variation. For generating prosodic hypotheses, the authors used models trained on the Boston Directions Corpus (BDC), which consists of elicited spontaneous speech and read transcripts.

Key Insights and Ideas

  • Prosody Matters for Pronunciation: The core idea is that the natural rhythm and emphasis of speech (prosody) directly influence how words are pronounced.
  • Prominence and Phrasing Reduce Variability: When words are emphasized (have pitch accent) or occur at sentence boundaries, they tend to be pronounced more carefully, leading to less variation. Conversely, deaccented words or words in the middle of phrases are more susceptible to reduction and coarticulation.
  • Hypothesized Prosody is Useful: Even though the prosodic annotations weren't directly available in the Buckeye Corpus, using predicted prosodic events significantly improved the pronunciation models.
  • Potential for Word Classing: The authors found preliminary evidence that accent patterns might be useful for distinguishing between content words and function words.

Key Results

The paper reports significant reductions in pronunciation model perplexity when incorporating prosodic features.

  • When considering all prosodic features (pitch accent, phrase-final boundary, phrase-initial boundary), the perplexity dropped from a baseline of 5.416 to 4.619 on the full Buckeye Corpus.
  • Even when compared against random binary features (used as a control to account for the addition of parameters), the prosodic features still yielded better performance.
  • Leave-one-speaker-out cross-validation also showed substantial improvements in perplexity, indicating the robustness of the findings across different speakers.
  • Accented tokens had an average of 3.32 pronunciations, while deaccented words had an average of 4.78, supporting the hypothesis of more careful pronunciation for accented words.
  • Words preceding or following phrase boundaries also showed fewer average pronunciations compared to those in the middle of phrases.

Limitations and Future Work

  • Sensitivity to Unseen Data: The current model is sensitive to out-of-vocabulary words and unseen pronunciations, as it relies on observed counts without smoothing.
  • Control for Added Parameters: While random binary features were used as a control, the authors acknowledge that the prosodic variables might be providing extra parameters that contribute to the perplexity reduction.
  • Future Work: The authors plan to integrate these findings into Automatic Speech Recognition (ASR) and Keyword Search systems to evaluate their impact on word error rates. They also intend to expand their evaluation to other corpora (like Switchboard), explore more sophisticated pronunciation models, and investigate the use of prosodic hypotheses in languages other than English. They also aim to further explore the use of prosodic information for unsupervised word-classing.

Summary updated

BibTeX
@inproceedings{rosenberg2012using,
  author = {Andrew Rosenberg},
  title = {{Using prominence and phrasing predictions to improve weighted dictionary pronunciation models}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure