TL;DR
This paper presents a novel multilingual Transformer model for grapheme-to-phoneme (G2P) conversion that significantly outperforms monolingual baselines across 15 languages. By treating G2P as a multilingual translation task and employing an ensemble of models, they achieve state-of-the-art results, demonstrating the power of shared representations for low-resource languages.
What Problem Does This Paper Address?
Grapheme-to-phoneme (G2P) conversion, the process of mapping written text to its phonetic pronunciation, is a fundamental task for speech recognition and synthesis systems. However, building effective G2P models is challenging, especially for languages with limited training data (low-resource languages). Traditional approaches often require separate models for each language, which is inefficient and doesn't leverage cross-lingual similarities. This paper tackles the challenge of building a single, highly effective G2P system for multiple languages, even with scarce data for each.
Methods
The core of the proposed solution is a multilingual Transformer model. Instead of training 15 separate models, they train a single Transformer model on data from all 15 languages simultaneously. To achieve this, they prepend a language code to each input grapheme sequence, effectively signaling to the model which language it's processing. This approach is inspired by successful multilingual machine translation systems, hypothesizing that shared representations across languages will lead to better performance, especially for low-resource scenarios.
They also explore a self-training approach to further augment their training data. This involves using their trained models to predict phonemes for unlabeled Wikipedia text. High-confidence predictions are then used as "silver" labeled data to re-train the models.
Finally, they employ an ensemble of models. This ensemble is constructed by averaging the predictions of multiple checkpoints from models trained with different random seeds. This technique is known to improve robustness and performance.
Datasets
The primary dataset used is from the SIGMORPHON 2020 Shared Task 1. This dataset consists of approximately 4,050 gold-labeled grapheme-phoneme pairs for each of 15 languages. The data was extracted from Wiktionary using the WikiPron library.
For data augmentation, they utilize Wikipedia dumps from 12 of the 15 languages (excluding Adyghe, Japanese, and Vietnamese due to specific reasons). This unlabeled text is used to generate "silver" training data through self-training.
Key Insights and Ideas
- Multilingualism as a Solution for Low-Resource G2P: The paper strongly suggests that treating G2P as a multilingual translation problem, rather than a collection of monolingual tasks, is a highly effective strategy, particularly when data is scarce.
- Shared Representations are Powerful: The success of the multilingual model indicates that learning shared phonetic representations across languages significantly benefits G2P conversion, even for languages with different writing systems.
- Ensembling Boosts Performance: Combining predictions from multiple model checkpoints (an ensemble) consistently outperforms individual models, highlighting the benefit of aggregating diverse predictions.
- Self-training's Limited Impact (in this study): While self-training is a promising technique for data augmentation, the authors found it did not significantly improve results in this specific context, possibly due to the limited scale of the augmented data or the inherent difficulty of the task.
Key Results
The proposed multilingual Transformer ensemble achieved impressive results on the SIGMORPHON 2020 Shared Task 1:
- Development Set: Achieved a Word Error Rate (WER) of 14.83% and a Phoneme Error Rate (PER) of 3.41%.
- Blind Test Set: Achieved a WER of 14.99% and a PER of 3.30%.
These results represent a significant improvement over the organizer-provided baselines, which had an average WER of 22.00% and PER of 4.92% for their Transformer baseline. The multilingual approach effectively halved the error rates compared to monolingual models.
Limitations and Future Work
- Self-training Scale: The authors acknowledge that the self-training approach did not yield the expected improvements, attributing this to the limited scale of the augmented data and time constraints. They suggest that scaling up the self-training process with more Wikipedia data could lead to better results.
- Ensemble Size: While their ensemble of four checkpoints was effective, they propose exploring larger ensembles, potentially including fully trained models (e.g., those trained for 200K steps).
- Writing System Correlation: Contrary to their initial hypothesis, they did not find a strong correlation between shared writing systems and improved performance. However, they noted that languages using the Cyrillic alphabet (Adyghe and Bulgarian) were among the worst-performing, suggesting potential areas for further investigation.
Summary updated