TL;DR
This paper explores using Wiktionary, a crowdsourced online dictionary, as a readily available source for automatically extracting phonetic pronunciations. The authors demonstrate that Wiktionary can significantly improve pronunciation dictionary coverage, especially for proper nouns, and can lead to performance gains in automatic speech recognition (ASR) systems, particularly for languages like Spanish.
What Problem Does This Paper Address?
Building robust automatic speech recognition (ASR) and speech synthesis systems for new languages or domains is a costly and time-consuming endeavor, largely due to the need for comprehensive pronunciation dictionaries. Manually creating these dictionaries is a bottleneck. While the World Wide Web offers a vast amount of text data, extracting accurate phonetic information from it for ASR system development has been challenging. This paper investigates whether Wiktionary, with its extensive collection of IPA (International Phonetic Alphabet) notations, can serve as a practical and cost-effective solution to rapidly generate or augment pronunciation dictionaries, thereby accelerating language adaptation for ASR systems.
Methods
The core of the paper's methodology involves an "Automatic Dictionary Extraction Tool" designed to parse Wiktionary. This tool takes a vocabulary list and, for each word, searches the corresponding Wiktionary page. It handles case variations to find pages even for misspelled words. The tool then extracts the first available IPA notation found on the page.
To evaluate the extracted pronunciations, two main checks were performed:
- Quantity Check: This assessed the percentage of words in given word lists (including domain-specific words and international city/country names) for which Wiktionary provided phonetic notations.
- Quality Check:
- Identical Pronunciations: Wiktionary pronunciations were compared against existing, manually verified pronunciations from the GlobalPhone project to determine the overlap.
- Impact on ASR Performance: The extracted pronunciations (after filtering out identical ones) were added to existing ASR system dictionaries. The performance of these augmented ASR systems was then evaluated using word error rate (WER) and class separability measures.
Datasets
- Wiktionary: Used as the primary source for extracting phonetic pronunciations. The paper specifically analyzed English, French, German, and Spanish editions.
- GlobalPhone Project: Provided existing pronunciation dictionaries for English, French, German, and Spanish, used as a baseline for comparison and for evaluating the impact of Wiktionary pronunciations on ASR systems. These dictionaries were built from national newspaper texts.
- International City and Country Names: A custom list was used to specifically test the coverage of proper nouns.
Key Insights and Ideas
- Wiktionary's Potential for Proper Nouns: The study found that Wiktionary is surprisingly effective at capturing pronunciations for proper nouns (like city and country names), which are notoriously difficult to generate using rule-based systems. This is a significant advantage for adapting ASR to real-world data.
- Language-Specific Performance: The effectiveness of Wiktionary varied significantly by language. French Wiktionary, for instance, showed exceptionally high coverage, while English Wiktionary performed less impressively, suggesting that the quantity and quality of phonetic annotations are not uniform across languages.
- Crowdsourcing Quality Variance: While crowdsourced data like Wiktionary is abundant, its quality can be inconsistent. The paper highlights that directly incorporating all Wiktionary pronunciations didn't always improve ASR performance, and in some cases (English, German), it led to degradation. This suggests a need for careful filtering or selection of pronunciations.
- The Value of Variants: Even when pronunciations weren't identical to existing ones, adding them as variants sometimes improved ASR performance (e.g., Spanish), indicating that Wiktionary can provide useful alternative pronunciations that capture natural speech variations.
Key Results
- Pronunciation Coverage: French Wiktionary achieved the highest coverage, with 93.27% of domain words having phonetic notations. For international city and country names, French coverage was 30.16% and 76.12%, respectively.
- Identical Pronunciations: French Wiktionary had the highest percentage of identical pronunciations (73.99%) compared to GlobalPhone.
- ASR Performance Improvement:
- System1 (adding all Wiktionary pronunciations): French and Spanish systems saw a decrease in WER (0.79% and 1.76% relative improvement, respectively). English and German systems showed degradation.
- System2 (using only Wiktionary pronunciations that were also present in training data): Spanish ASR performance improved significantly with a 7.22% relative word error rate reduction. French also saw a slight improvement (1.17%).
Limitations and Future Work
- Inconsistent Quality: The paper acknowledges that the quality of Wiktionary pronunciations can be inconsistent, and some may be erroneous or not relevant to the target language.
- Single IPA Notation Extraction: The current tool only extracts the first IPA notation found, potentially missing other valid pronunciations or variants.
- Language-Specific Data Differences: The varying performance across languages points to differences in the quantity and quality of phonetic data available in different Wiktionary editions.
- Future Work: The authors plan to explore other Wiktionary language editions, improve their extraction methods to handle multiple IPA notations, and develop techniques to categorize and filter pronunciations, including those not in the target language. They also aim to investigate other web sources like Wikipedia for pronunciation data and explore using Wiktionary for training grapheme-to-phoneme models.
Summary updated