Introduction
Hybrid Automatic Speech Recognition (ASR) systems, which integrate Deep Neural Networks (DNNs) with Hidden Markov Models (HMMs), typically operate using fixed word-level vocabularies. While effective for common language, these systems are notoriously brittle when encountering underrepresented words—terms that appear rarely or not at all in training data.
Named Entities (NEs), such as specific street names or local landmarks, are frequently underrepresented. In diverse linguistic environments like Singapore, street names of Malay or Indian origin may be rare in general English training corpora. Failure to recognize these entities, such as "Boon Lay," directly impacts downstream Natural Language Understanding (NLU) tasks, rendering voice assistants ineffective for navigation or local searches. This research explores a multi-pronged strategy to improve the recognition of these rare and out-of-vocabulary (OOV) entities without degrading overall system performance.
Datasets and Evaluation
The researchers utilized two primary datasets to evaluate their methods:
- SEAME: A 101-hour corpus of Mandarin-English code-switching speech used for primary training and general evaluation.
- SG-streets: A specialized evaluation set consisting of one hour of recordings where speakers read passages containing local Singaporean street names.
From the SG-streets set, 393 underrepresented NEs were identified and categorized into rare-NE (appearing fewer than 10 times in training) and oov-NE (zero appearances in training).
Methodology
The proposed solution involves four complementary techniques designed to enhance the ASR system's ability to handle rare words:
- Graphemic Lexicon: Traditional phonetic lexicons rely on Grapheme-to-Phoneme (G2P) models, which are often error-prone for rare names. By using a context- and position-dependent graphemic lexicon, the system treats letters as the basic units of sound, bypassing G2P errors.
- Neural LM with Letter-Based Features: A Recurrent Neural Network Language Model (RNNLM) is employed that incorporates subword (letter n-gram) features. This allows the model to better generalize the context of a word even if the word itself is rare.
- Embedding Matrix Augmentation: To improve the mathematical representation of rare words, their vector embeddings are "enriched" by averaging them with the embeddings of semantically or phonetically similar frequent words.
- Word Lattice Boosting: As a post-processing step, the system indexes the word lattice (the graph of possible word sequences) and manually increases the likelihood scores for paths containing target Named Entities.
Key Results and Analysis
The combination of these methods yielded dramatic improvements in Named Entity Word Error Rate (NE-WER) on the SG-streets dataset. As shown in the experimental results, the system achieved a significant reduction in errors for both rare and OOV entities.
Key findings include:
- Significant NE-WER Reduction: The overall NE-WER improved from a baseline of 57.3% down to 24.7%.
- Rare and OOV Gains: Error rates for rare-NEs dropped from 15.2% to 3.5%, while OOV-NE error rates were slashed from 100% to 45.1%.
- Graphemic vs. Phonetic: The results indicate that graphemic lexicons often outperform phonetic ones when expanding the system to include new words, suggesting that G2P error propagation is a major bottleneck.
- System Stability: Crucially, the overall Word Error Rate (WER) on standard test sets (test<sub>man</sub> and test<sub>sge</sub>) remained stable, proving that rare word recognition can be improved without compromising general accuracy.
Conclusion
By combining graphemic lexicons, embedding augmentation, and lattice boosting, the researchers achieved a substantial relative improvement in named entity recognition. While recognizing words the model has never seen (OOV) remains a challenge, this multi-pronged approach provides a robust framework for making hybrid ASR systems more inclusive of local and underrepresented vocabulary without requiring additional labeled training data.
Summary updated