Introduction
End-to-End (E2E) Automatic Speech Recognition (ASR) models have achieved remarkable performance in transcribing general speech. However, they frequently struggle with "out-of-vocabulary" (OOV) or rare terms, such as technical jargon, proper nouns, or contact names. Because these models rely heavily on the frequency of terms in their training data, they often misinterpret rare, context-specific words as phonetically similar common words.
To address this, researchers have explored "deep biasing" techniques that allow users to provide a "bias list"—a set of contextually relevant phrases—to the model at inference time. This paper introduces a novel deep biasing approach that utilizes a bias phrase index loss, special contextual tokens, and a boosted beam search algorithm to significantly improve the recognition of these targeted phrases without requiring a full retraining of the ASR model.
Proposed Methodology
The proposed system is built upon a Conformer-based ASR architecture and introduces three primary innovations to integrate contextual information effectively.
1. Bias Encoder and Cross-Attention
The architecture incorporates a dedicated Bias Encoder that transforms the provided bias list into mathematical embeddings. During the decoding process, a cross-attention layer allows the model to attend to both the acoustic features and these bias embeddings simultaneously. This ensures that the model is "aware" of the relevant phrases while processing the audio signal.
2. Bias Phrase Index Loss ($L_{bidx}$) and Special Tokens
To force the model to utilize the bias list more effectively, the authors introduced a Bias Phrase Index Loss. Unlike previous methods that updated biasing mechanisms indirectly, this loss function explicitly trains the model to predict which index in the bias list corresponds to the current transcription.
Furthermore, the model is trained to use special tokens—<sob> (Start of Bias) and <eob> (End of Bias)—to encapsulate phrases from the list. These tokens act as anchors, helping the model distinguish between general vocabulary and specific terms from the bias list, thereby reducing confusion during the decoding process.
3. Bias Phrase Boosted (BPB) Beam Search
During inference, the researchers implemented a Bias Phrase Boosted (BPB) Beam Search. Standard beam search algorithms typically select the most likely next token based on overall probability. BPB Beam Search, however, utilizes the probability scores from the bias index to "nudge" or boost the scores of tokens that belong to a detected bias phrase. This mechanism ensures that even if a rare word has a lower initial probability than a common word, the contextual relevance provided by the bias list allows it to be correctly identified.
Experimental Setup
The effectiveness of the proposed method was evaluated on two major benchmarks:
- Librispeech-960: A standard English corpus consisting of 960 hours of speech.
- In-house Japanese Dataset: A comprehensive dataset combining internal data (93 hours), the Corpus of Spontaneous Japanese (581 hours), and the ATR Japanese speech database (181 hours).
The researchers tested the system's scalability by varying the size of the bias list ($N$) from 100 to 1,000 phrases.
Results and Discussion
The proposed method demonstrated consistent improvements over both the standard hybrid CTC/attention baseline and existing state-of-the-art biasing methods like CPPNet.
Performance Gains
- Librispeech-100 (Preliminary): The full system reduced the Bias-Word Error Rate (B-WER) from 30.71% in the baseline to 17.93%.
- Librispeech-960: On the "test-clean" set with a 100-phrase bias list, the proposed model achieved a B-WER of 6.0%, nearly halving the baseline's 11.7%.
- Japanese Dataset: Using a list of 203 phrases, the Bias-Character Error Rate (B-CER) dropped from 22.32% to 13.16%.
Scalability and Robustness
A common failure point for biasing systems is "list fatigue," where performance degrades as the bias list grows larger. However, the BPB Beam Search proved robust; even when the list was expanded to 1,000 phrases, the error rates remained significantly lower than the baseline. This suggests the attention-based mechanism effectively filters out irrelevant candidates even in high-density contexts.
Conclusion and Future Work
This research presents a powerful framework for contextualized ASR. By combining explicit index loss with a boosted search algorithm, the model achieves superior accuracy on rare and context-specific terms.
While highly effective, the authors noted a slight performance degradation when the bias list is empty ($N=0$), suggesting the model develops a dependency on the biasing mechanism. Additionally, the cross-attention layers and bias encoder introduce some computational overhead. Future research will focus on optimizing these components for real-time performance on edge devices and expanding the methodology to tonal languages and languages with high out-of-vocabulary rates.
Summary updated