Linguistic Knowledge in Multilingual Grapheme-to-Phoneme Conversion

Roger Yu-Hsiang Lo, Garrett NicolaiACL 2021

TL;DR

This paper tackles the challenging task of converting written text (graphemes) into their spoken sounds (phonemes), particularly in low-resource languages where data is scarce. The UBC Linguistics team developed a system that, by incorporating linguistic knowledge about syllable structure and specifically penalizing vowel errors, significantly outperformed baseline models and all other submissions in the SIGMORPHON 2021 challenge.

What Problem Does This Paper Address?

Grapheme-to-phoneme (G2P) conversion is a fundamental step in many speech technology applications, like text-to-speech systems. It involves mapping written characters to their corresponding phonetic representations, often using the International Phonetic Alphabet (IPA). While G2P is well-studied for high-resource languages, it becomes significantly more difficult for low-resource languages due to the limited availability of training data. This scarcity makes it challenging for standard machine learning models to learn the complex relationships between spelling and pronunciation. The SIGMORPHON 2021 shared task specifically addressed this by providing a low-resource setting with only 800 training examples per language, pushing researchers to find more efficient and data-aware methods.

Methods

The UBC Linguistics team built upon a strong baseline neural transducer model, which uses an imitation learning paradigm to predict edit actions (copy, delete, insert, substitute) for converting graphemes to phonemes. They explored two main linguistic-inspired approaches:

  1. Augmenting Data with Unsupervised Syllable Boundaries: This method aimed to leverage the inherent structure of language by identifying syllable boundaries. They developed a heuristic approach to detect vowels in both input and output, align graphemes to phonemes, and identify valid onset and coda consonant clusters. This information was then used to segment words into syllables, creating new input sequences with explicit syllable boundary markers. The idea was that syllable structure could provide useful contextual information for G2P conversion.
  2. Penalizing Vowel and Diacritic Errors: This approach was driven by an analysis of the baseline model's errors, which revealed a significant number of mistakes related to vowels and diacritics (like vowel length or pitch accent). To address this, they modified the model's training objective by adding explicit penalties for incorrect vowel and diacritic predictions. This modification aimed to force the model to pay more attention to these crucial phonetic elements.

Datasets

The study utilized data from the SIGMORPHON 2021 Grapheme-to-Phoneme Shared Task, specifically focusing on the low-resource subtask. This dataset comprised ten languages: Adyghe, Modern Greek, Icelandic, Italian, Khmer, Latvian, Maltese (Latin script), Romanian, Slovene, and the South Wales dialect of Welsh. For each language, the dataset was split into:

  • Training set: 800 labeled grapheme-phoneme pairs.
  • Development set: 100 labeled grapheme-phoneme pairs.
  • Blind test set: 100 labeled grapheme-phoneme pairs.

The data was extracted from Wikitionary using WikiPron and then filtered and downsampled.

Key Insights and Ideas

  • Linguistic knowledge is crucial for low-resource G2P: Even simple linguistic insights, like syllable structure and common error patterns, can significantly boost performance when data is limited.
  • Vowel errors are a major bottleneck: The analysis revealed that many G2P errors, especially in certain languages, stem from misinterpreting vowel quality, length, or diacritics.
  • Targeted penalties work: Directly penalizing specific types of errors (like vowel mistakes) during training can be more effective than relying solely on the standard sequence prediction objective.
  • Syllable boundaries might not always help: While intuitively appealing, the explicit marking of syllable boundaries did not improve performance in this specific low-resource setting, suggesting that the unsupervised annotation might have been too noisy or that the information was redundant for the model architecture.

Key Results

The team's second system (VP), which incorporated penalties for vowel and diacritic errors, achieved the best results. It outperformed the baseline model and all other submitted systems in the low-resource setting, achieving the lowest macro-average Word Error Rate (WER). For instance, the average WER for their VP system was 24.1, compared to the baseline's 25.1. The vowel penalty was found to have a significantly greater impact than the diacritic penalty.

Limitations and Future Work

The paper acknowledges that the first system, which used syllable boundaries, did not outperform the baseline. The authors speculate that this could be due to the unsupervised nature of the syllable annotation being too noisy, or that the added variability from syllable boundaries didn't provide enough useful information for the current neural transducer architecture. They suggest that future work could explore more sophisticated methods for syllable boundary detection or investigate other linguistic units. The error analyses also highlight that challenges remain, particularly with suprasegmental features like vowel length and pitch accent, which are often not explicitly marked in orthography, suggesting avenues for further research in modeling these aspects.

Summary updated

BibTeX
@inproceedings{lo2021linguistic,
  author = {Roger Yu-Hsiang Lo and Garrett Nicolai},
  title = {{Linguistic Knowledge in Multilingual Grapheme-to-Phoneme Conversion}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure