TL;DR
This paper introduces a novel framework for reversible grapheme-to-phoneme (G2P) and phoneme-to-grapheme (P2G) conversion, crucial for spoken dialogue systems to automatically acquire new words. By combining linguistic syllable structure with statistical modeling, they achieve high accuracy in predicting spellings from pronunciations, even for words not seen during training.
What Problem Does This Paper Address?
Spoken dialogue systems are becoming more sophisticated, but their ability to learn and adapt is limited by how well they can handle new words. When a user says a word the system doesn't know, it needs to figure out both its spelling and its pronunciation to potentially add it to its knowledge base. This requires a robust system that can go from sound to letters (P2G) and letters to sound (G2P) reliably. English, with its many loanwords and inconsistent spelling-sound relationships, makes this a particularly challenging task. Existing methods often struggle with out-of-vocabulary (OOV) words, hindering the seamless acquisition of new knowledge.
Methods
The authors propose a two-stage approach:
- Linguistic Modeling of Syllable Structure: They first parse words into their underlying syllable structure using a context-free grammar. This structure is decomposed into "spellneme" units, which are symbolic tags that encode both graphemic (spelling) and phonemic (sound) information. This linguistic decomposition helps to capture the internal structure of words.
- Statistical Modeling with Spellnemes: A statistical n-gram language model is then trained on these spellneme units. This model learns the probabilistic relationships between sequences of spellnemes. For the P2G task, the system expands a given phonemic transcript into a graph of possible spellneme sequences, which is then pruned by the n-gram model to produce the most likely spelling. For G2P, the process is reversed.
A key aspect is the iterative process of deriving these spellnemes and correcting parse failures, ensuring the grammar and subword units accurately represent the lexicon.
Datasets
- Training Data: A lexicon of 140,000 words was used to derive the subword units and train the statistical models. A subset of 55,159 nouns from the Comlex lexicon was specifically used for training the n-gram model.
- Evaluation Data: The experiments were conducted on a set of nearly 5000 nouns and proper nouns from the Phonebook corpus. This set was chosen because it contained many nouns not in the original training lexicon, allowing for evaluation on both in-vocabulary (IV) and out-of-vocabulary (OOV) words.
Key Insights and Ideas
- Syllable Structure is Key: Decomposing words into linguistic units like onsets and rhymes, rather than treating whole syllables, provides a more granular and generalizable representation.
- "Spellnemes" Bridge the Gap: The novel "spellneme" units, which combine graphemic and phonemic information, are crucial for creating a reversible mapping. They act as a common symbolic representation that both sound and spelling can be mapped to.
- Iterative Refinement is Essential: The manual correction of parse failures and augmentation of rules is vital for building a high-quality lexicon of subword units, which directly impacts the performance of the statistical models.
- OOV Words are a Major Challenge: The performance gap between IV and OOV words highlights the difficulty of handling unseen words, underscoring the need for robust OOV handling mechanisms.
Key Results
The paper reports strong performance on sound-to-letter (P2G) experiments for the Phonebook nouns, even for OOV words:
- With an N-best list of 30 hypotheses, the system achieved a very low failure rate (1.6% for OOV words).
- The mean depth of the correct spelling in the N-best list was 2.07 for OOV words, compared to 1.15 for IV words.
- Automatic augmentation of the training set with automatically generated baseforms for OOV words significantly improved performance, reducing the mean depth to 1.54.
- Manual correction of these automatically generated baseforms further boosted performance, achieving a mean depth of 1.13, comparable to the IV subset.
Limitations and Future Work
- Limitations: The current work focuses on nouns and proper nouns. The performance on other word classes might differ. The evaluation was primarily on sound-to-letter, with letter-to-sound being described as straightforward.
- Future Work: The authors plan to evaluate the effectiveness of this model for automatic new word acquisition in spoken dialogue systems. They also aim to explore the G2P direction more thoroughly.
Summary updated