Evaluation of many-to-many alignment algorithm by automatic pronunciation annotation using web text mining

Keigo Kubo, Hiromichi Kawanami, Hiroshi Saruwatari, Kiyohiro ShikanoInterspeech 2012

TL;DR

This paper introduces an improved many-to-many alignment algorithm for automatically annotating pronunciations of out-of-vocabulary (OOV) words. By incorporating a "city block distance" and refining parameter estimation to handle irrelevant mappings and deletions, the new method significantly boosts pronunciation annotation accuracy for OOV words, crucial for applications like voice search.

What Problem Does This Paper Address?

Modern applications like voice search need to handle a vast and ever-growing vocabulary, including proper nouns and newly coined terms that aren't in standard dictionaries. These are known as out-of-vocabulary (OOV) words. To process these words, we need to know their pronunciation. While we can easily find OOV words from web text, their pronunciations are often unknown. Automatically generating these pronunciations (grapheme-to-phoneme conversion) is a key challenge. A critical component of these conversion methods is aligning the written form (graphemes) with the spoken form (phonemes). Traditional alignment methods, like one-to-one mapping, struggle with the complexity of pronunciation, while existing many-to-many approaches can sometimes overfit to common patterns, hindering their ability to generalize to new, unseen OOV words. This paper aims to improve the generalization capability of many-to-many alignment for OOV words.

Methods

The core of this work is an enhanced "many-to-many" alignment algorithm. Unlike one-to-one alignments, many-to-many allows a sequence of graphemes to map to a sequence of phonemes, and vice-versa. The authors build upon the "joint multigram" approach, which treats alignments as a product of probabilities of individual mappings.

The key innovations are:

  • City Block Distance: To prevent the algorithm from favoring overly long mappings (which can hurt generalization), they introduce the "city block distance" (sum of the lengths of grapheme and phoneme sequences in a mapping) into the probability calculation. This equalizes the advantage of longer mappings, encouraging shorter, more generalizable ones.
  • Improved Parameter Estimation:
    • Handling Deletion Characters: They address the issue where the algorithm might incorrectly learn parameters for "deletion characters" (phonemes that don't correspond to any grapheme, or vice-versa). This is done using "n-best Viterbi training," which focuses on more promising alignments and a modified EM algorithm that initially prohibits deletion characters during parameter estimation.
  • Handling Irrelevant Mappings: They tackle the problem of "irrelevant mappings" – mappings that are learned due to coincidental occurrences in the training data but don't represent true pronunciation rules. This is addressed by merging mappings that have limited contextual support with their neighbors, assuming that mappings with broader contextual evidence are more likely to be correct.

Datasets

The paper uses the following datasets:

  • Training Data: A combination of Japanese Kanji dictionaries (Wnn, Sanseido, Naist Japanese dictionary) and an English-Japanese dictionary (Eijiro), totaling approximately 350,000 words.
  • Test Data: Approximately 2,000 query words collected from Yahoo! search rankings, Google Trends, and Goo keyword rankings. These queries were specifically selected because they couldn't be annotated by a conventional tool (Kytea), implying they are likely OOV or have complex pronunciations.

Key Insights and Ideas

  • The Trade-off in Alignment Length: The paper highlights that while longer mappings might seem more descriptive, they can actually harm the algorithm's ability to generalize to new OOV words. The "city block distance" is an elegant way to balance mapping length and probability.
  • The Importance of Robust Parameter Estimation: Simply having a many-to-many structure isn't enough. The way parameters are learned is crucial. The techniques to handle deletion characters and irrelevant mappings are essential for building a truly effective pronunciation annotator.
  • Web Text Mining as a Realistic Evaluation: Using web text mining to automatically annotate pronunciations provides a practical and scalable way to evaluate pronunciation models on real-world OOV words, which is more representative than relying solely on curated dictionaries.

Key Results

The proposed "city+del+merge" algorithm demonstrated superior performance in terms of Recall (generalization ability for OOV words) compared to the conventional "joint multigram" approach.

  • Recall: The proposed method achieved a Recall of 90.31%, outperforming the best joint method (joint 1-6) by about 1.9 percentage points. This difference was statistically significant.
  • Precision: While the proposed method had slightly lower Precision (occurrence of irrelevant mappings) than the best joint method (joint 3-6), the difference was not statistically significant, indicating that the improvement in Recall did not come at a significant cost to accuracy.
  • F-value: The proposed method also achieved the highest F-value (a combined measure of Recall and Precision), outperforming the best joint method (joint 2-3) by about 1.9 percentage points.

Limitations and Future Work

  • Limitations: The paper notes that the "n-best Viterbi training" in the proposed methods uses only 2-best alignments. The hypothesis that deletion characters continuously appear in pronunciation hypotheses is prohibited.
  • Future Work: The paper doesn't explicitly detail future work, but the success of the proposed enhancements suggests further exploration into more sophisticated parameter estimation techniques and potentially larger-scale web text mining for even more robust pronunciation annotation. The authors also mention that the optimal parameters for the city block distance might be language-dependent, hinting at potential cross-lingual applications or further tuning.

Summary updated

BibTeX
@inproceedings{kubo2012evaluation,
  author = {Keigo Kubo and Hiromichi Kawanami and Hiroshi Saruwatari and Kiyohiro Shikano},
  title = {{Evaluation of many-to-many alignment algorithm by automatic pronunciation annotation using web text mining}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure