Low-Resource G2P and P2G Conversion with Synthetic Training Data

Bradley Hauer, Amir Ahmad Habibi, Yixing Luan, Arnob Mallik, Grzegorz KondrakACL 2020

TL;DR

This paper tackles the challenge of low-resource grapheme-to-phoneme (G2P) and phoneme-to-grapheme (P2G) conversion by introducing a novel data augmentation technique that synthesizes training data from unannotated text. Their method leverages multiple existing G2P/P2G models to create more robust training examples, significantly improving performance in low-resource scenarios.

What Problem Does This Paper Address?

Converting written text (graphemes) into spoken sounds (phonemes), known as G2P, is crucial for applications like text-to-speech systems. The inverse task, P2G, is important for understanding spelling complexity. While high-resource languages have abundant data for training these models, many languages lack sufficient annotated data, making it difficult to build accurate G2P/P2G systems. This paper addresses this "low-resource" problem by proposing a method to generate synthetic training data, effectively boosting the performance of models when real data is scarce. This is particularly relevant for linguistic diversity and preserving endangered languages.

Methods

The core of the paper's contribution lies in its data augmentation method. The authors propose a way to synthesize additional training data for G2P conversion using unannotated text. The intuition is that a P2G model should be the inverse of a G2P model. Their approach involves:

  1. Training base G2P and P2G models: They use existing transduction systems (DTLM and FST) trained on the limited available data.
  2. Extracting words from unannotated text: They use large text corpora (like Wikipedia) to get a list of words.
  3. Synthesizing candidate training instances: They apply the base G2P model to these words to generate phoneme sequences.
  4. Filtering and validation: This is the crucial step. To avoid propagating errors, they apply several sanity checks:
  5. The G2P models must agree on the phoneme prediction.
  6. The corresponding P2G models must agree and, when applied to the predicted phonemes, recover the original grapheme word.
  7. Training on augmented data: The final model is trained on the combination of original and synthetically generated data.

They also utilize the DTLM (Discriminative String Transduction) system, a powerful sequence-to-sequence model that incorporates various features like context n-grams, target language models, and joint n-grams. For P2G, they leverage word counts and character language models to improve accuracy.

Datasets

The paper uses datasets from the SIGMORPHON 2020 Task 1: Multilingual Grapheme-to-Phoneme Conversion.

  • High-resource setting: 3600 training instances per language.
  • Low-resource setting: A newly created benchmark of 100 training instances per language, derived by selecting every 36th instance from the high-resource training set to ensure replicability.
  • P2G datasets: Created by simply swapping the grapheme and phoneme strings from the G2P task datasets.
  • Unannotated text corpora: Used for data augmentation, including Wikipedia dumps.

Key Insights and Ideas

  • Data augmentation is key for low-resource G2P/P2G: The paper strongly demonstrates that synthetic data generation can dramatically improve neural model performance when real data is scarce, without requiring architectural changes.
  • Cross-validation of G2P and P2G models is essential for data synthesis: The core idea of using the inverse relationship between G2P and P2G models to filter and validate synthetic data is a clever way to ensure data quality.
  • DTLM's robustness in low-resource settings: The DTLM system, even without augmentation, shows competitive performance in low-resource scenarios compared to more complex neural models like TRANSFORMER.
  • Leveraging monolingual text is powerful for P2G: The significant gains observed for P2G when using word counts and character language models highlight the importance of linguistic context.

Key Results

  • The data augmentation approach was successful in development experiments, reducing the average Word Error Rate (WER) on high-resource datasets from 17.0% to 16.0%.
  • In the low-resource setting, the data augmentation approach yielded an average WER improvement of over 35% compared to base TRANSFORMER models.
  • DTLM consistently outperformed FST and TRANSFORMER on P2G tasks, especially in the low-resource setting.
  • For G2P, DTLM also outperformed FST, though TRANSFORMER showed better results in the high-resource setting (without augmentation).

Limitations and Future Work

  • TRANSFORMER replication issues: The authors were unable to replicate the official results for the TRANSFORMER model, citing differences in implementation and hyperparameter settings.
  • Data augmentation limitations for certain scripts: The augmentation process produced fewer new instances for Vietnamese and Korean, which they attribute to the unusual characteristics of these scripts.
  • Potential for overfitting with augmentation: While successful in development, the augmentation did not yield consistent improvements on the official test results for high-resource settings, possibly due to limitations on training epochs or suboptimal tuning leading to overfitting.
  • Future work: The paper suggests that the data augmentation approach is a strong proof-of-concept for improving neural models in low-resource scenarios, implying further exploration and refinement of this technique. They also highlight the potential for applying this to P2G as well.

Summary updated

BibTeX
@inproceedings{hauer2020low,
  author = {Bradley Hauer and Amir Ahmad Habibi and Yixing Luan and Arnob Mallik and Grzegorz Kondrak},
  title = {{Low-Resource G2P and P2G Conversion with Synthetic Training Data}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
}
Figure