Abstract
Traditional Automatic Speech Recognition (ASR) systems rely on a fixed lexicon and word-level language models, creating a bottleneck for out-of-vocabulary (OOV) words. This paper demonstrates that character-based convolutional language models (ConvLMs) with large receptive fields can match the performance of word-based systems. We show that lexicon-free decoding not only achieves competitive Word Error Rates (WER) on standard benchmarks but also significantly outperforms lexicon-based systems on utterances containing OOV words.
The Lexicon Bottleneck
Most ASR systems are constrained by a dictionary. They use a lexicon—a predefined list of valid words—and a word-level language model to guide transcription. This architecture fails when encountering OOV words like new technical terms or proper nouns. In such cases, the system typically "hallucinates" the phonetically closest known word. While character-based systems avoid this constraint, they have historically struggled to capture the long-distance semantic context that whole words provide.
Methodology: Deep Character Context
The authors propose an end-to-end pipeline operating entirely at the character level, consisting of three core components:
- Acoustic Model (AM): A 1D Gated Convolutional Neural Network (GCNN) trained with the Auto Segmentation Criterion (ASG). It maps audio features to 31 graphemes (alphabet, punctuation, and a silence token).
- Language Model (LM): The "secret sauce" is the use of deep Convolutional LMs. By increasing the network depth, the authors achieved a receptive field of up to 81 characters. This allows the model to capture context comparable to traditional word-based models.
- Lexicon-Free Decoding: A modified beam-search decoder scores transcriptions using acoustic scores, character-LM scores, and penalties for word/silence length. Without a lexicon restriction, the system can generate any sequence of characters.
Performance and Accuracy
The system was evaluated on the Wall Street Journal (WSJ) and LibriSpeech datasets. On LibriSpeech, the lexicon-free GCNN-20B model demonstrated that it could match or even slightly exceed the performance of lexicon-constrained models.
On the WSJ dataset, the lexicon-free model achieved a 3.6% WER, matching the performance of the same model when constrained by a lexicon. Crucially, for utterances containing OOV words, the lexicon-free approach showed a massive relative improvement, achieving 18.3% WER compared to 22.2% for a standard word-based 4-gram model.
Efficiency and Decoding
A surprising finding was the efficiency of character-level decoding. Despite the lack of word-level constraints, character-based models require a smaller effective beam size to maintain accuracy.
While word-level decoders often need to track 2,000–8,000 hypotheses, the character-based model remains effective with a beam size around 2,000. This suggests that the character LM provides a strong enough signal to prune the search space effectively.
Conclusion
This research proves that lexicon-free, character-level modeling is a robust alternative to traditional word-based ASR. By leveraging large character contexts through deep convolutional architectures, these systems overcome the OOV problem without sacrificing accuracy on standard vocabulary. Future work may involve scaling to more complex writing systems and exploring Transformer-based architectures to further extend context.
Summary updated