SIGMORPHON 2022 Shared Task on Grapheme-to-Phoneme Conversion Submission Description: Sequence Labelling for G2P

Leander GirrbachSIGMORPHON 2023 · DOI

TL;DR

This paper explores sequence labeling methods for grapheme-to-phoneme (G2P) conversion, a crucial step in speech technology. The authors propose three novel sequence labeling approaches and find that while they don't outperform sequence-to-sequence models on high-resource tasks, they offer a compelling alternative for low-resource scenarios and provide more interpretable alignments.

What Problem Does This Paper Address?

Grapheme-to-phoneme (G2P) conversion is the process of transforming written text into its spoken sound representation. This is a fundamental task for many speech technologies, including text-to-speech (TTS) systems and automatic speech recognition (ASR). Traditionally, G2P has been tackled using sequence-to-sequence models, which are powerful but can be computationally expensive and lack direct interpretability. This paper investigates sequence labeling methods as an alternative, aiming to achieve comparable performance, especially in low-resource settings, while offering advantages like simpler inference and more transparent alignment between input graphemes and output phonemes. The SIGMORPHON 2022 shared task specifically focused on low-resource and cross-lingual G2P, highlighting the need for efficient and effective methods when data is scarce.

Methods

The authors propose and evaluate three sequence labeling methods for G2P:

  • Supervised Sequence Labelling: This method is inspired by prior work and involves an initial alignment step. They use a neuralized Expectation-Maximization (EM) aligner to learn grapheme and phoneme embeddings and then derive alignments. These alignments are then used to train a sequence labeling model. A key design choice is how to handle "many-to-one" alignments (multiple graphemes mapping to a single phoneme), which they convert to "one-to-one" cases by assigning the phoneme label to the first grapheme and a "deletion" label to subsequent graphemes.
  • Gram-CTC Sequence Labelling: This approach builds on the CTC (Connectionist Temporal Classification) framework, which is commonly used in speech recognition. Instead of predicting individual phonemes, Gram-CTC predicts "phoneme ngrams" (sequences of phonemes) from graphemes. A whitelist of allowed phoneme ngrams, derived from the alignments of the supervised method, is used to constrain the predictions. This method does not rely on explicit pre-computed alignments for training.
  • Inverse-Scatter-CTC Sequence Labelling: This method also leverages CTC but takes a different approach to handle length differences. For each grapheme, it predicts a fixed number of phoneme unigrams (single phonemes), ensuring that the predicted phoneme sequence is always longer than the grapheme sequence. This allows the use of standard CTC loss. The authors found that predicting 7 phonemes per grapheme worked well for most languages, with a slightly higher value needed for Persian.

All models utilize a single-layer BiLSTM architecture. For the transfer track, they share embeddings and the LSTM encoder but use separate classification layers for the target and transfer languages.

Datasets

The experiments were conducted on data from the SIGMORPHON 2022 Shared Task on Grapheme-to-Phoneme Conversion (Low-Resource and Cross-Lingual). The task included:

  • High Resource Track: 1000 grapheme-phoneme pairs for 10 languages.
  • Low Resource Track: 100 grapheme-phoneme pairs for 10 languages.
  • Transfer Track: 100 grapheme-phoneme pairs in the target language and 1000 pairs in a related transfer language.

A common test set of 100 words and a development set were provided for each track and language.

Key Insights and Ideas

  • Sequence Labeling for G2P is Viable, Especially in Low Resource Settings: While sequence-to-sequence models often dominate high-resource G2P tasks, sequence labeling methods demonstrate competitive performance, particularly when data is scarce. This is a significant finding for practical applications where large datasets are not always available.
  • Interpretability is a Key Advantage: A major selling point of sequence labeling is the direct alignment it provides between input graphemes and output phonemes. The authors show that these alignments can be easily inspected, aiding in error analysis and model understanding.
  • Gram-CTC Offers a Data-Efficient Approach: By learning alignments implicitly and using a whitelist of phoneme ngrams, Gram-CTC avoids the need for explicit, potentially noisy, pre-computed alignments, making it more robust.
  • Inverse-Scatter-CTC's Flexible Length Handling: This method's approach of predicting a fixed, larger number of phonemes per grapheme effectively addresses the length mismatch problem inherent in G2P, allowing for standard CTC training.
  • Language-Specific Performance: The paper highlights that no single model is universally superior across all languages and tracks, emphasizing the need for language-specific tuning and evaluation.

Key Results

  • High and Low Resource: Gram-CTC generally outperforms the supervised sequence labeling method, achieving a lower macro-average Word Error Rate (WER). The difference is particularly pronounced for Thai in the high-resource setting.
  • Transfer Task: For the transfer task, supervised training shows improvement over its low-resource performance, suggesting that transfer data is beneficial when used with the right model. Gram-CTC, while still outperforming supervised training on many languages, shows a slight decrease in performance compared to its low-resource setting.
  • Inverse-Scatter-CTC: This method shows competitive performance, sometimes slightly better than Gram-CTC, but without decisive advantages. The choice of the parameter $\tau$ (number of phonemes predicted per grapheme) impacts performance, with $\tau=4$ showing good results for the transfer track.
  • Model Inspection: The Inverse-Scatter-CTC model successfully learned reasonable alignments for common German graphemes, demonstrating the interpretability advantage.

Limitations and Future Work

  • Sequence-to-Sequence Dominance in High Resource: The paper acknowledges that sequence-to-sequence models generally perform better in high-resource scenarios.
  • Handling of Complex Graphemes: While the Inverse-Scatter-CTC model showed good alignment capabilities, it struggled with predicting multiple phonemes from a single grapheme in some cases (e.g., German "x").
  • Future Work Suggestions: The authors suggest that incorporating predefined mappings of graphemes to phonemes could further restrict predictions and improve accuracy. They also propose that error analysis, as facilitated by their models, can guide future data augmentation efforts (e.g., adding more examples of rare graphemes).

Summary updated

BibTeX
@inproceedings{girrbach2023sigmorphon,
  author = {Leander Girrbach},
  title = {{SIGMORPHON 2022 Shared Task on Grapheme-to-Phoneme Conversion Submission Description: Sequence Labelling for G2P}},
  booktitle = {SIGMORPHON},
  year = {2023},
  doi = {10.18653/v1/2023.sigmorphon-1.28},
}
Figure