Introduction
In Large Vocabulary Continuous Speech Recognition (LVCSR), systems are typically trained on hundreds of hours of transcribed audio. However, a significant challenge arises when users search for Out-of-Training (OOT) terms—words that exist in the system's dictionary but were rare or absent in the original training data. While the system may know the pronunciation, the Language Model (LM) assigns these terms a very low probability due to a lack of contextual examples. This paper presents a semi-supervised approach to "fill in the holes" of an LM using automatic transcripts of unlabeled audio, achieving substantial performance gains without new human labels.
Proposed Methodology
The researchers explored three primary strategies to increase the visibility of OOT terms:
- Unigram Probability Clamping: A "brute force" method that artificially boosts a search term's probability in the LM. While this improves recall, it often leads to "hallucinations" in noisy audio, hurting precision.
- Semi-Supervised Language Modeling: The core contribution involves using an existing LVCSR system to transcribe 1,700 hours of unlabeled audio. By extracting $n$-grams from consensus networks (which represent multiple hypotheses) rather than just the "1-best" transcript, the researchers provided the LM with the necessary context for OOT terms.
- Directed Transcription (Active Learning): For scenarios with a limited human budget, the authors proposed "directing" transcribers to verify only short snippets where the semi-supervised system predicts a keyword might exist.
Key Insights: LM vs. AM
A critical finding of this research is that the Language Model benefits significantly more from semi-supervised data than the Acoustic Model (AM). LMs suffer from a lack of "parameter sharing"; if a word is missing from training text, the model has zero information about its context. In contrast, the AM can often rely on sub-word phonetic models shared across different words. Narrowing the focus to rare keywords allows for massive gains that are typically obscured when measuring average Word Error Rate (WER) across an entire corpus.
Results and Efficiency
The study was conducted on English Conversational Telephone Speech (CTS), using the Switchboard corpus for training and the Fisher collection for unlabeled evaluation. The baseline system achieved an Area Under the Curve (AUC) of 0.45 for OOT terms. The semi-supervised method raised this to 0.589, capturing 70% of the potential gain that would have been achieved through full manual transcription (AUC 0.648).
Furthermore, the "Directed Transcription" approach proved to be 125 times more cost-effective than undirected manual transcription. As shown in the performance analysis, spending just 270 hours on targeted verification yielded results nearly identical to transcribing the full 34,000-hour equivalent of the Fisher corpus.
Conclusion and Future Work
This semi-supervised approach effectively addresses the "long tail" of rare keywords in speech recognition. While the LM saw dramatic improvements, the AM remains a potential bottleneck for rare phoneme sequences. Future work may involve applying similar directed techniques to acoustic modeling. Additionally, while demonstrated on English CTS, the robustness of this method in lower-resource languages remains an area for further investigation.
Summary updated