When we read text, we often operate under the illusion that letters are simply direct instructions for speech sounds. In a perfectly transparent (or "shallow") orthography, this is true: every written character (grapheme) maps to exactly one spoken sound (phoneme). However, natural languages rarely cooperate. Over centuries of historical drift, script borrowing, and morphological evolution, writing systems develop orthographic depth.
In deep or opaque orthographies, the relationship between spelling and pronunciation becomes an intricate, many-to-many puzzle. As Sunayana Sitaram et al. point out in Universal grapheme-based speech synthesis, these inconsistent spelling-to-sound correspondences prevent simple character-based models from accurately predicting speech sounds. Resolving this orthographic opacity is one of the most persistent bottlenecks in modern speech recognition (ASR), text-to-speech (TTS), and cross-script transliteration.
Why Standard Sequence Models Stumble
Modern machine learning excels at sequence-to-sequence mapping, but deep orthographies introduce structural friction that standard architectures cannot easily smooth over.
When an automatic speech recognition system attempts to transcribe audio directly into text at the character level, it must resolve severe phonetic ambiguities. As Kevin Kilgour et al. demonstrate in The 2013 KIT IWSLT speech-to-text systems for German and English, predicting characters directly from audio yields significantly higher error rates in deep orthographies compared to phoneme-based approaches. Without an integrated understanding of grammar and orthography, end-to-end models struggle to resolve homophones or correctly spell words with non-phonetic structures—a fundamental challenge highlighted by Alex Graves et al. in Towards End-To-End Speech Recognition with Recurrent Neural Networks.
Conversely, in text-to-speech synthesis, grapheme-only models face a steep uphill battle. They are forced to implicitly memorize irregular pronunciation rules hidden within contextual context clues. This mapping complexity often necessitates massive amounts of training data just to achieve basic pronunciation accuracy (Heiga Zen et al., PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS).
The Dimensions of Orthographic Complexity
To understand why the grapheme-to-phoneme (G2P) interface breaks down, we must look at the specific linguistic mechanisms that cause orthographic opacity:
- Phonetic Under-specification: Many writing systems systematically omit critical acoustic information. Abjads, for example, omit short vowels. In other cases, such as the highly under-resourced language Chaha, complex phonological features like labialization or palatalization are entirely unrepresented in the standard script, creating severe ambiguity for acoustic-to-grapheme models (Tessfu Geteye Fantaye et al., Investigation of Automatic Speech Recognition Systems via the Multilingual Deep Neural Network Modeling Methods for a Very Low-Resource Language, Chaha).
- Morphophonemic Invariance: Writing systems often prioritize preserving the visual identity of a meaningful root word (morphology) over reflecting its actual sound changes (phonology). Walter Daelemans notes in GRAFON: A Grapheme-to-Phoneme Conversion System for Dutch that intricate morphology and compounding make it exceptionally difficult for traditional systems to map text to sound accurately. Similarly, morphographic systems like Korean maintain static spellings even when grammatical boundaries trigger dramatic phonetic transformations (Jinsik Lee et al., Hybrid approach to grapheme to phoneme conversion for Korean).
- Etymological Layering: Proper names and borrowed loanwords carry the orthographic rules of their origin languages, creating pockets of irregularity within a host language. As Aditya Bhargava et al. explain in How do you pronounce your name? Improving G2P with transliterations, English name strings are notoriously opaque because identical character sequences map to completely different sounds depending on the name's etymological background.
Architectural Strategies for Overcoming Opacity
To bridge the gap between text and sound, researchers have moved away from naive character-sequence modeling, opting instead for architectures that explicitly account for phonological structure.
Joint Representation Spaces
Instead of forcing a model to choose between text or audio tokens, modern architectures embed both into a shared representation space. In PnG BERT, Heiga Zen et al. designed a neural encoder that takes both phonemes and graphemes as input during pre-training. By enforcing a word-level alignment strategy, the model learns to anchor the volatile, irregular relationships of written characters to their stable phonetic equivalents, closing the gap between synthetic speech and natural human prosody.
Articulatory Feature Vectors
Treating phonemes as discrete, unrelated tokens (like vocabulary words) discards the underlying physical reality of speech. Human speech sounds are produced by physical configurations of the vocal tract (e.g., voicing, nasalization, place of articulation).
David R. Mortensen et al. introduced PanPhon (PanPhon: A Resource for Mapping IPA Segments to Articulatory Feature Vectors), a toolkit that maps International Phonetic Alphabet (IPA) segments into sub-phonemic articulatory feature vectors. By breaking sounds down into these universal physical dimensions, models can generalize across ambiguous orthographies and low-resource settings far more effectively than they could using abstract character tokens.
[ Written Text ] ---> [ Grapheme-to-Phoneme Model ] ---> [ IPA Segment ]
|
v
[ Articulatory Features ]
• Voicing: [+voiced]
• Nasal: [-nasal]
• Place: [+alveolar]
Linguistically-Informed Tokenization
Standard sub-word tokenizers (such as Byte-Pair Encoding) slice text based purely on statistical frequency, which frequently destroys phonetic coherence. To fix this, researchers introduce linguistically constrained tokenization boundaries.
For instance, in Automatic Speech Recognition in Sanskrit: A New Speech Corpus and Modelling Insights, Devaraja Adiga et al. utilize a "vowel split" sub-word modeling technique. By forcing token boundaries to align with phonetic transformations, they prevent the acoustic-to-language model interface from becoming muddled by complex word compounding. Similarly, Chen Li et al. segment text into phonetically meaningful "character blocks" (like ght or tion) in Improving Text Normalization using Character-Blocks Based Models and System Combination, allowing models to capture local orthographic patterns before they are obscured by global sequence variations.
Multilingual Transfer and Ensembles
When a language lacks an expert-curated pronunciation dictionary, its orthographic depth can feel insurmountable. However, phonological systems are often shared across language families, even if their writing systems differ.
By treating G2P conversion as a multilingual translation task, models can borrow structural strength from high-resource languages to navigate the ambiguities of low-resource ones. Both Kaili Vesik et al. (One Model to Pronounce Them All: Multilingual Grapheme-to-Phoneme Conversion With a Transformer Ensemble) and Omnia ElSaadany et al. (Grapheme-to-Phoneme Conversion with a Multilingual Transformer Model) demonstrate that a single multilingual Transformer model can match or exceed the performance of isolated monolingual models. The shared attention mechanisms within these Transformers learn to map diverse script types to a unified phonetic space, proving that the underlying mechanics of human speech can successfully untangle the historical eccentricities of the written word.
Key Foundational Work
A Procedure of an Automatic Grapheme-to-Phoneme Transformation of German (1982) — contributed 1 concept elements to this direction.