TL;DR
This paper tackles the challenge of accurately predicting word pronunciations (grapheme-to-phoneme conversion) by leveraging morphological information. They introduce novel methods for incorporating morphological class and lemma information into deep neural networks, achieving significant improvements, especially for morphologically rich languages and low-resource settings.
What Problem Does This Paper Address?
Accurately converting written words into their phonetic representations (pronunciation prediction or G2P) is crucial for speech recognition and synthesis systems. While deep learning models have made strides, they often struggle with the nuances of pronunciation, particularly for morphologically related words. These words, like "cops," "cogs," and "courses" in English, share a common root but have different pronunciations due to morphological rules. Manually creating comprehensive pronunciation dictionaries for all word forms is infeasible, especially for morphologically rich languages. Existing rule-based or purely data-driven G2P models can miss these morphological patterns, leading to errors. This paper aims to bridge this gap by explicitly incorporating morphological information into neural G2P models to improve their accuracy.
Methods
The authors propose two primary ways to inject morphological information into neural G2P models:
- Morphological Category: The morphological class of a word (e.g., noun, verb, plural) is encoded as an ordinal integer and appended to the input spelling. This provides a concise signal about the word's grammatical role.
- Lemma Spelling and Pronunciation: The lemma (base form) of the word, along with its spelling and pronunciation, is also appended to the input. This allows the model to leverage the known pronunciation of the root word, which can be particularly helpful for irregular forms or when the surface form is ambiguous.
They experiment with three neural network architectures:
- Bidirectional LSTM (BiLSTM): A standard recurrent neural network architecture.
- BiLSTM with Attention (BiLSTM+Attn): Enhances the BiLSTM by allowing it to focus on relevant parts of the input sequence.
- Transformer: A more recent architecture that relies entirely on self-attention mechanisms.
The models are implemented using the OpenNMT toolkit. They also explore transfer learning, using models trained on high-resource languages to improve performance on low-resource languages.
Datasets
The study uses pronunciation lexicons and morphological lexicons for five languages: American English, French, Russian, Spanish, and Hungarian. For evaluation, 20% of the pronunciation lexicons are set aside. They also mention experiments on larger datasets (10⁵-10⁶ annotated inflections) for English, German, Spanish, and Russian. For transfer learning experiments, they illustrate with Finnish and Portuguese, noting that these have fewer Wiktionary annotations compared to the primary languages.
Key Insights and Ideas
- Morphology is Key for G2P: Explicitly incorporating morphological information significantly boosts pronunciation prediction accuracy, especially for complex cases.
- Lemma Power: Providing lemma spelling and pronunciation is particularly effective, helping to resolve ambiguities and handle irregular forms.
- BiLSTM+Attn Shines: The BiLSTM with attention architecture appears to be the most effective among the tested models, balancing performance and complexity.
- Low-Resource Gains: Morphology injection offers substantial improvements in low-resource settings, sometimes yielding gains comparable to transfer learning from a high-resource language.
- Language-Specific Benefits: The gains from morphology are more pronounced in morphologically rich languages (like Russian) and languages with complex grapheme-phoneme mappings (like English).
Key Results
The paper reports significant reductions in Word Error Rate (WER) across languages when morphology is injected. For instance, in English, the BiLSTM+Attn model with both morphology class and lemma injection achieved a relative error reduction of approximately 25%. For Russian, morphological category injection alone led to a similar relative error reduction. In transfer learning experiments, adding lemma information to a BiLSTM+Attn model for Finnish and Portuguese resulted in substantial improvements, bringing their performance closer to that of higher-resource languages. The BiLSTM+Attn model consistently outperformed the baseline BiLSTM and Transformer models, especially when dealing with longer sequences introduced by morphological information.
Limitations and Future Work
The authors acknowledge that their current work focuses primarily on inflectional morphology. They suggest future work could extend this to derivation and compounding, which are more complex and might require unsupervised learning or dictionary augmentation. They also note the importance of explainability in neural models and suggest extending morphological feature explanations. Finally, they propose developing non-neural approaches for morphology injection, though they acknowledge neural methods are currently state-of-the-art. The potential for using this approach to automatically annotate pronunciation lexicons for low-resource languages is highlighted as a promising application.
Summary updated