Unified Language-Independent DNN-Based G2P Converter

Markéta Jůzová, Daniel Tihelka, Jakub VítInterspeech 2019

TL;DR

This paper introduces a novel, unified deep learning framework for converting written text (graphemes) into their phonetic pronunciations (phonemes), designed to work across multiple languages without language-specific tuning. By training on entire phrases rather than individual words, their model effectively captures complex phonetic interactions, achieving competitive or superior performance compared to traditional methods.

What Problem Does This Paper Address?

Converting written text into its spoken pronunciation (Grapheme-to-Phoneme, or G2P) is a fundamental step in many speech technologies, like text-to-speech (TTS) systems. Traditionally, G2P systems rely on either large dictionaries of word-pronunciation pairs (good for irregular languages like English) or rule-based systems (better for regular languages like Czech or Russian). However, these approaches have limitations: dictionaries struggle with new or inflected words and cross-word phonetic changes, while rule-based systems can be complex to develop and maintain, and often miss exceptions. A significant challenge is handling "cross-word assimilation," where the pronunciation of a word is influenced by the words around it, which is difficult to model with word-by-word processing. This paper aims to overcome these limitations by proposing a single, language-independent model that can handle these complexities by processing text in larger chunks.

Methods

The core of the proposed solution is a deep neural network (DNN) employing an encoder-decoder architecture, similar to those used in machine translation.

  • Input Processing: Instead of individual words, the model takes sequences of words (phrases) as input. These are then converted into sequences of symbols of equal length using a padding symbol.
  • Embedding Layer: Each grapheme (character) is transformed into a vector representation.
  • Bidirectional LSTM Encoder: This layer processes the embedded graphemes to learn within-word phonetic relationships. It then generates a fixed-length representation for each word, effectively creating "word embeddings."
  • Cross-Word Interaction Layer: This layer, which can be either another bidirectional LSTM or a convolutional layer, is designed to learn relationships between adjacent words, crucial for capturing cross-word assimilation.
  • LSTM Decoder: This component takes the word embeddings and generates the output phoneme sequence. It uses a "feedback loop" where the previously generated phoneme is fed back as input to predict the next phoneme, allowing it to learn sequential dependencies.
  • Output: A final linear projection with softmax activation produces the phoneme probabilities.

The authors experimented with two main DNN structures: one using a bidirectional LSTM for cross-word interactions and another using a convolutional layer. They also tested different lengths of word sequences (3, 5, and 10 words) for training.

Datasets

The paper utilizes proprietary datasets for English, Russian, and Czech. These datasets consist of hundreds of thousands of phrases, each paired with its corresponding phonetic transcription. The transcriptions were generated using a combination of large dictionaries and expert systems, with a portion randomly checked by phonetic experts to ensure quality and tune the systems. The authors used 90% of these datasets for training and 10% for testing. No new benchmarks were introduced; instead, the performance was compared against existing baseline G2P systems on these established datasets.

Key Insights and Ideas

  • Phrase-level processing is key: The most significant insight is that processing text in phrases, rather than isolated words, is highly effective for capturing complex phonetic phenomena like cross-word assimilation. This approach inherently handles inter- and cross-word influences without needing explicit language-specific rules or topologies.
  • Language Independence is Achievable: The framework demonstrates that a single DNN architecture can perform well across languages with very different phonetic properties (e.g., irregular English vs. regular Czech) without requiring language-specific tuning.
  • DNNs can learn implicit rules: The model learns to handle phenomena like stress placement in Russian and irregular pronunciations in English implicitly through its training on phrase-level data, reducing the need for handcrafted rules.
  • Context matters for homographs: While the model's primary advantage is handling phonetic assimilation, the authors suggest that wider context (beyond just immediate neighbors) might help resolve homographs (words spelled the same but pronounced differently), similar to how Part-of-Speech (POS) tagging is used in some TTS systems.

Key Results

The proposed DNN-based G2P converter generally outperformed the baseline G2P systems (which combine rules and dictionaries) across all tested languages in terms of word accuracy.

  • English: The best DNN model achieved a word accuracy of 94.17%, significantly higher than the baseline's 81.90%.
  • Czech: The DNN models achieved word accuracies close to 99%, comparable to the baseline's 98.74%, likely due to Czech's relatively regular pronunciation.
  • Russian: The DNN models showed improvement over the baseline (81.36% vs. 77.56% word accuracy), though the results were lower than for English and Czech, which the authors attribute to the baseline's own difficulties with Russian stress placement.
  • Sequence Length: Using word sequences of length 5 or 10 yielded similar, strong results, suggesting that a moderate context window is sufficient, and very long sequences are not necessarily beneficial.

Limitations and Future Work

  • Data Dependency: The primary limitation acknowledged is the need for a large corpus of text paired with correct phonetic transcriptions. This data can be costly and time-consuming to acquire.
  • Longer Words: The DNN model showed a tendency to make more errors on longer words (10 or more graphemes), suggesting that while phrase-level context helps, very long word structures still pose a challenge.
  • Future Directions: The authors plan to incorporate additional linguistic features, such as Part-of-Speech (POS) tags, into the model to further improve the transcription of homographs. They also intend to continue experimenting with different parameter settings and architectures for their phrase-level G2P converter.

Summary updated

BibTeX
@inproceedings{juzova2019unified,
  author = {Markéta Jůzová and Daniel Tihelka and Jakub Vít},
  title = {{Unified Language-Independent DNN-Based G2P Converter}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure