Introduction
Grapheme-to-Phoneme (G2P) conversion is the essential task of mapping written text (graphemes) to its phonetic representation, typically using the International Phonetic Alphabet (IPA). This process is particularly challenging for languages with "opaque" orthographies, where the relationship between spelling and sound is inconsistent or complex.
The CLUZH team participated in the SIGMORPHON 2020 Shared Task on Multilingual G2P, securing second place overall. Their approach involved adapting a neural transducer—originally designed for morphological inflection—to handle the unique challenges of G2P, where input and output alphabets are often entirely disjoint.
Methodology: Neural Transduction with Expert Policies
The core of the CLUZH system is a neural transducer that generates phoneme sequences through a series of explicit edit actions: insertions, deletions, and substitutions. Unlike previous models that relied heavily on "copying" input characters (common in morphology), this system was optimized for translation between different symbol sets.
Imitation Learning and the SED Expert
The model is trained using Imitation Learning (IL), where it learns to mimic an "expert policy" that demonstrates the optimal sequence of edits. To provide high-quality demonstrations, the authors implemented a Stochastic Edit Distance (SED) model.
As shown in Figure 1, the SED is a probabilistic Weighted Finite-State Transducer (WFST). It learns the specific costs of mapping graphemes to phonemes (e.g., 'g' to /g/) directly from the training data. This learned distance metric allows the expert policy to guide the neural learner along more intuitive and accurate paths than a standard Levenshtein distance would allow.
Key Innovations
- Substitution Edits: Explicit substitution actions were introduced to minimize the total number of edit steps, improving efficiency and accuracy.
- Unicode Normalization: For scripts like Korean (Hangul), the team used NFKD normalization to decompose complex syllable blocks into constituent parts, significantly simplifying the mapping task.
- Ensembling: To mitigate the risk of overfitting on small development sets (450 examples per language), the team employed a majority-vote ensemble of 11 models for each language, which reduced error rates by 16% compared to single-model averages.
Experimental Results
The system was evaluated on the SIGMORPHON 2020 Shared Task I dataset, covering 15 diverse languages including Indo-European, Uralic, Kartvelian, and Japonic families.
As detailed in Figure 2, the CLUZH system achieved an average Word Error Rate (WER) of 14.13% and a Phoneme Error Rate (PER) of 2.82%. The system was particularly strong in Hindi, Hungarian, Icelandic, and Lithuanian, where it tied for first place. It significantly outperformed the official LSTM and Transformer baselines across nearly all metrics.
The results also highlight the varying difficulty of the task: while the model achieved near-perfect results for Vietnamese (1.56% WER), it faced significant challenges with Adyghe (27.11% WER) and Korean (28.44% WER), likely due to complex phonology and highly non-phonetic scripts.
Discussion and Future Work
The success of the CLUZH system underscores the importance of the expert policy in imitation learning. By replacing fixed distance metrics with a learned SED model, the researchers effectively bridged the gap between disjoint alphabets.
Future improvements could address contextual ambiguities, such as homographs in French, by incorporating morphological or lexical tags. Additionally, integrating external dictionaries or unlabelled data could help the model better handle rare words and foreign borrowings.
Reference: Makarov, P., & Clematide, S. (2020). CLUZH at SIGMORPHON 2020 Shared Task on Multilingual Grapheme-to-Phoneme Conversion. Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, 171–176.
Summary updated