Abstract
There is an implicit assumption that traditional hybrid approaches for automatic speech recognition (ASR) cannot directly model graphemes and need to rely on phonetic lexicons to get competitive performance, especially on English which has poor grapheme-phoneme correspondence. In this work, we show for the first time that, on English, hybrid ASR systems can in fact model graphemes effectively by leveraging tied context-dependent graphemes, i.e., chenones. Our chenone-based systems significantly outperform equivalent senone baselines by 4.5% to 11.1% relative on three different English datasets. Our results on Librispeech are state-of-the-art compared to other hybrid approaches and competitive with previously published end-to-end numbers. Further analysis shows that chenones can better utilize powerful acoustic models and large training data, and require context- and position-dependent modeling to work well. Chenone-based systems also outperform senone baselines on proper noun and rare word recognition, an area where the latter is traditionally thought to have an advantage. Our work provides an alternative for end-to-end ASR and establishes that hybrid systems can be improved by dropping the reliance on phonetic knowledge.
Introduction
Hybrid ASR systems, which combine Hidden Markov Models (HMMs) with Deep Neural Networks (DNNs), have historically relied on senones—tied context-dependent states of phones—as their basic modeling unit. This reliance stems from the irregular relationship between spelling and sound in English. Consequently, building these systems has traditionally required significant linguistic expertise to create and maintain phonetic lexicons.
While modern End-to-End (E2E) models have successfully transitioned to graphemic units, they often require vast amounts of training data to learn these relationships from scratch. This research explores whether the data efficiency of hybrid systems can be combined with the simplicity of graphemic modeling, effectively bypassing the "phonetic tax" of expert-curated dictionaries.
The Chenone Approach
The core innovation of this work is the replacement of senones with chenones. To overcome the inherent ambiguity of English spelling, the authors implement three critical modeling strategies:
- Context Dependency: Utilizing tri-graphemes (looking at characters before and after) to resolve phonetic ambiguity.
- Position Dependency: Explicitly marking characters that occur at word boundaries (e.g., "_WB").
- Casing Information: Preserving uppercase letters to distinguish abbreviations and proper nouns, providing implicit cues about pronunciation.
The system employs a standard hybrid pipeline, utilizing Latency Controlled Bidirectional LSTM (LC-BLSTM) acoustic models trained with Cross-Entropy and Lattice-Free MMI (LF-MMI) criteria.
Experimental Evaluation
The performance of chenones was evaluated across three distinct datasets: LibriSpeech (960 hours), a large-scale Video dataset (13.7k hours), and an Assistant dataset (12.5k hours).
As shown in Figure 2, chenone-based systems (Gr) consistently outperformed senone-based baselines (Ph) across all tasks. On LibriSpeech, the system achieved a Word Error Rate (WER) of 3.2% on test-clean and 7.6% on test-other, setting a new benchmark for hybrid systems and rivaling much larger E2E models.
Key Insights and Analysis
- Scaling and Capacity: Chenones demonstrate a superior ability to exploit larger acoustic models and more training data. As model capacity (e.g., 6x1000 LC-BLSTM) increases, the performance gap in favor of chenones widens significantly.
- Rare Word Recognition: Traditionally, phonetic lexicons were thought to be essential for recognizing rare words. Surprisingly, chenones performed better on proper nouns and "long-tail" vocabulary. This is likely because they avoid the errors inherent in automated Grapheme-to-Phoneme (G2P) converters used to generate pronunciations for out-of-vocabulary words in traditional systems.
- Criticality of Context: Ablation studies revealed that position dependency is vital for graphemic modeling; removing word boundary markers resulted in a 14.7% relative degradation in WER.
Conclusion
By rethinking the role of graphemes in hybrid ASR, this research demonstrates that phonetic knowledge is not a prerequisite for high-performance English speech recognition. Chenones offer a simplified, more accurate, and more scalable alternative to traditional phone-based modeling, bridging the gap between hybrid efficiency and the simplicity of graphemic systems.
Summary updated