For decades, speech recognition and synthesis systems relied on a hidden intermediary to bridge the gap between text and sound: the phoneme. To map written characters (graphemes) to spoken sounds, speech engines required an expert-curated pronunciation dictionary or lexicon. However, this reliance introduces what Duc Le et al. term the "phonetic tax" in From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition. Manually constructing these dictionaries requires immense labor from specialized linguists, creating a severe bottleneck that hinders scalability across the world's thousands of languages (Universal grapheme-based speech synthesis).
To scale speech technology globally, researchers are moving away from static, human-engineered lexicons. By shifting toward direct graphemic modeling, unsupervised phonetic discovery, and cross-lingual transfer, modern systems allow neural networks to learn the relationships between text and speech directly from data.
Bypassing the Dictionary via Graphemic Modeling
The most direct way to eliminate the lexicon bottleneck is to remove the phonetic layer entirely. Instead of forcing a model to learn a two-step mapping ($\text{Grapheme} \rightarrow \text{Phoneme} \rightarrow \text{Audio}$), architectures can map graphemes directly to acoustic signals.
This approach works because deep neural networks are highly capable of learning latent alignments. In Towards End-To-End Speech Recognition with Recurrent Neural Networks, Alex Graves et al. demonstrated that by optimizing a Connectionist Temporal Classification (CTC) objective, a single network can transcribe audio directly at the character level. The network bypasses the need for a hand-designed dictionary by discovering its own internal representations of how character sequences correspond to temporal audio patterns.
For languages with irregular spelling rules like English, critics historically argued that explicit phonetic guidance was necessary. However, Duc Le et al. challenged this by introducing chenones—context-dependent graphemes. By tying together character states based on their surrounding context, hybrid speech recognition systems can actually outperform traditional phone-based models. The model learns the phonetic nuances implicitly from the data, eliminating the manual curation burden.
Data-Driven Discovery of Pronunciations
When an explicit mapping between text and sound is still required for downstream tasks, researchers use data-driven optimization to discover pronunciations automatically. Rather than relying on static rule sets, these systems treat the lexicon as a learnable component of the network.
In Joint Learning of Phonetic Units and Word Pronunciations for ASR, Chia-ying Lee et al. framed this as a hierarchical Bayesian problem. Their system jointly learns the phonetic inventory of a language and the letter-to-sound mappings simultaneously from transcribed audio, approaching the performance of expert-crafted dictionaries.
To handle low-resource scenarios or regional dialects where transcriptions do not exist, unsupervised discovery becomes essential. Systems can extract continuous speech representations directly from unlabelled audio to map word similarities. For example, the authors of Automatic Text Pronunciation Correlation Generation and Application for Contextual Biasing utilized speech embeddings and Dynamic Time Warping to learn how similar different words sound without any human-written dictionary. Similarly, unsupervised frameworks like wav2vec-U 2.0 (Towards End-to-end Unsupervised Speech Recognition) and acoustic segment models (An acoustic segment modeling approach to automatic language identification) completely bypass human phonetic definitions by discovering stable, data-driven sound units directly from raw audio.
Cross-Lingual Transfer and Transliteration
For the vast majority of the world's languages, creating a standalone pronunciation dictionary is impractical due to a lack of data and linguistic expertise (The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech). The core intuition for solving this is knowledge reuse: borrowing phonetic or orthographic structures from high-resource languages.
One method is to exploit commonalities across writing systems. As shown in Developing Pronunciation Models in New Languages Faster by Exploiting Common Grapheme-to-Phoneme Correspondences Across Languages, grapheme-to-phoneme (G2P) models can bootstrap performance in a new language by transferring rules from phonetically or orthographically related languages. In text-to-speech synthesis, Wei-Ping Huang demonstrated in Few Shot Cross-Lingual TTS Using Transferable Phoneme Embedding that a shared codebook module can map phonemes into a global space, allowing a model to synthesize intelligible speech in an entirely unseen language with under a minute of audio data.
Alternatively, systems can map diverse scripts into a single, unified orthographic space via universal transliteration. In Multilingual Query-by-Example KWS for Indian Languages using Transliteration, Kirandevraj R et al. bypassed the need for ten separate phonetic dictionaries by transliterating multiple distinct Indic scripts into a single Devanagari representation. This unified text target allowed a self-supervised model to learn cross-lingual acoustic-text correlations efficiently, completely circumventing the traditional manual lexicon bottleneck.
Key Foundational Work
Automatic generation of Korean pronunciation variants by multistage applications of phonological rules (1998) — contributed 1 concept elements to this direction.