Low-resource grapheme-to-phoneme mapping with phonetically-conditioned transfer

Michael HammondACL 2023

TL;DR

This paper tackles the challenge of converting written words to their phonetic pronunciations in languages with very little data. The authors found that traditional neural network approaches struggled, but a classical Hidden Markov Model (HMM) system, combined with clever data augmentation and careful selection of transfer data from related languages, significantly improved performance.

What Problem Does This Paper Address?

Accurately converting written text (graphemes) into their spoken sounds (phonemes) is crucial for many natural language processing tasks, like speech synthesis and recognition. However, building these "grapheme-to-phoneme" (G2P) systems is difficult for languages with limited training data. The SIGMORPHON 2022 challenge presented a scenario with only 100 training examples per language, plus some additional data from a related language. This paper explores how to build effective G2P systems under such extreme low-resource conditions, focusing on non-neural methods and data augmentation.

Methods

The authors experimented with two main approaches:

  • Initial Neural Approaches: They started with a standard transformer-based neural network, similar to those used in previous G2P challenges. This involved a 512-element embedding layer, six encoder/decoder layers, and an 8-head attention mechanism.
  • HMM-based Approach: Recognizing the limitations of neural networks with scarce data, they switched to a classical HMM-based system called Phonetisaurus. This system utilizes weighted finite-state transducers (WFSTs) for alignment and decoding.
  • Data Augmentation: Several techniques were employed to expand the training data:
  • Substring Recombination: Inspired by prior work, this method recombines plausible substrings from word beginnings and ends, aiming to create more realistic training examples.
  • Transfer Data Filtering: To leverage the "related language" data more effectively, they implemented several filtering strategies:
  • Including only transfer pairs where phonetic segments were present in the target language's inventory.
  • Further restricting to pairs where phonetic bigrams from the transfer language also occurred in the target language.
  • Adding orthographic overlap criteria.
  • Combining phonetic bigram sharing with substring recombination.

Datasets

The primary datasets used were from the SIGMORPHON 2022 grapheme-to-phoneme conversion challenge. This included:

  • Target Languages: 10 languages with 100 training pairs and 100 development pairs each.
  • Transfer Languages: For each target language, up to 1000 additional training pairs from a related language were provided.
  • Test Sets: 100 additional pairs for each language were used for final evaluation.
  • CMU Pronouncing Dictionary: Used for initial experiments to understand data requirements for the neural system.

Key Insights and Ideas

  • Neural Networks Struggle with Extreme Low-Resource Settings: The initial transformer model performed very poorly with only 100 training pairs, highlighting the limitations of data-hungry models in such scenarios.
  • HMMs Offer a Stronger Baseline: Classical HMM-based systems like Phonetisaurus provided a much better starting point for low-resource G2P.
  • Transfer Data Quality Matters: Simply adding all transfer data from a related language didn't always help and sometimes hurt performance. Carefully selecting transfer data based on phonetic similarity to the target language was crucial.
  • Phonetic Similarity is Key: Restricting transfer data to pairs with shared phonetic segments and, more importantly, shared phonetic bigrams, led to significant improvements.
  • Orthographic Similarity is Also Beneficial: While phonetic similarity was more impactful, orthographic overlap also showed some positive effects.
  • Data Augmentation Complements Filtering: The substring recombination technique, when applied on top of filtered transfer data, provided an additional boost in performance.

Key Results

  • The initial transformer model achieved very high Word Error Rates (WER), indicating poor performance. For example, on the 100-pair condition with transfer data, the average WER was 51.63%.
  • The Phonetisaurus HMM system, out-of-the-box, performed significantly better, with an average WER of 51.63% for the 100-pair condition with transfer data.
  • By carefully filtering transfer data based on phonetic bigram overlap and applying substring recombination, the authors achieved a mean validation WER of 76.65% on the test set (Table 5, phbg+1900 column).
  • On the test set, their final system (phonetically-restricted transfer data + substring recombination) achieved a mean WER of 75.69% for the full 1000 pairs.

Limitations and Future Work

  • Limitations: The paper acknowledges that the neural approaches performed poorly due to the extreme data scarcity. The effectiveness of transfer data filtering was highly dependent on the specific language pairs.
  • Future Work: The authors suggest that further exploration into more sophisticated methods for selecting and utilizing transfer data, as well as more advanced data augmentation techniques, could lead to even better results. The success of combining phonetic similarity with substring recombination also points towards promising avenues for future research.

Summary updated

BibTeX
@inproceedings{hammond2023low,
  author = {Michael Hammond},
  title = {{Low-resource grapheme-to-phoneme mapping with phonetically-conditioned transfer}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure