Towards Contextual Spelling Correction for Customization of End-to-End Speech Recognition Systems

Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Veljko Miljanic, Sheng Zhao, Hosam Khalil

Introduction

Modern end-to-end (E2E) automatic speech recognition (ASR) systems have achieved remarkable performance across general domains. However, they frequently struggle with "contextual biasing"—the ability to correctly recognize rare or user-specific terms such as contact names, unique locations, or domain-specific jargon. For instance, a voice assistant might transcribe "Call Jessa" as "Call and" because the word "and" is statistically more probable in the model's general training data.

Traditional approaches to this problem, such as shallow fusion with a secondary language model or modifying the internal ASR architecture, face significant hurdles. Shallow fusion is difficult to tune for sub-word units, while internal modifications are computationally expensive and often fail to scale when a user has thousands of potential bias terms (e.g., a large contact list). This paper proposes a novel, "plug-in" post-processing approach that applies a contextual spelling correction layer to the ASR output, offering a scalable and efficient solution for system customization.

Methodology

The researchers introduce a post-processing module that takes the initial ASR hypothesis and a list of context phrases to generate a corrected transcript. They evaluate two distinct architectures for this task:

1. Contextual Spelling Correction (CSC)

The CSC model is an autoregressive (AR) transformer. It generates the corrected text one token at a time by attending to both the original ASR hypothesis and the provided context list. While effective, the sequential nature of autoregressive generation can be slow for real-time applications.

2. Fast Contextual Spelling Correction (FCSC)

To address the latency of the AR model, the authors developed the FCSC, a non-autoregressive (NAR) model. Rather than generating text from scratch, the FCSC treats correction as a classification task. It employs two primary mechanisms:

  • BILO Tagging: The model predicts "Beginning, Inside, Last, Outside" tags to identify which segments of the ASR output correspond to a phrase in the context list.
  • Similarity Layer: Once a segment is identified for correction, the model uses a similarity layer to select the most likely phrase index from the provided bias list.

Scaling and Robustness

To manage large-scale context lists, the system utilizes a dual-ranking filtering mechanism: a relevance ranker (rRanker) based on edit distance and a preference ranker (pRanker) based on frequency. Furthermore, the authors introduced a controllable confidence threshold ($s^o$). This allows developers to tune the system's aggressiveness, ensuring the model does not "hallucinate" names or over-correct general speech when no context terms are actually present.

Experimental Setup

Due to the scarcity of real-world audio paired with exhaustive context lists, the authors utilized a synthetic Text-to-Speech (TTS) pipeline for training. They generated audio for 1 million names and 6 million proper nouns, passing the synthetic audio through an ASR system to create "noisy" hypothesis-reference pairs for the correction model to learn from.

Evaluation was conducted across several datasets:

  • Name Sets: Personal assistant development and blind test sets.
  • Random8k: An "anti-context" set used to monitor and prevent over-correction of general speech.
  • Domain-Specific Sets: A "Text Editor" set (commands) and a "Medical" set (specialized health terms) to test the model's ability to generalize beyond its training data.

Key Results and Insights

The proposed FCSC model demonstrated significant improvements in both accuracy and efficiency:

  • Accuracy: The FCSC achieved a 51% relative Word Error Rate (WER) reduction on the Name test set compared to a baseline Transformer-Transducer (T-T) system.
  • Efficiency: The non-autoregressive FCSC model is 43.2% smaller and 2.1x faster during inference than the autoregressive CSC version.
  • Generalization: The model showed remarkable adaptability. On the Medical dataset—which featured vocabulary entirely different from the training data—the system still provided a 38.6% WER reduction. This suggests the model learned the underlying logic of error correction rather than simply memorizing specific terms.

Conclusion

This "plug-in" philosophy allows for significant ASR customization without the need to retrain or modify the core ASR engine. By treating contextual biasing as a post-processing classification task, the FCSC model provides a high-speed, scalable solution for domain-specific applications.

Future work will focus on two areas: acoustic grounding, which would allow the model to utilize audio features to distinguish between similar-sounding names, and streaming capabilities, to enable the non-autoregressive solution to function in real-time, low-latency environments.

Summary updated

BibTeX
@article{wang2022towards,
  author = {Xiaoqiang Wang and Yanqing Liu and Jinyu Li and Veljko Miljanic and Sheng Zhao and Hosam Khalil},
  title = {{Towards Contextual Spelling Correction for Customization of End-to-End Speech Recognition Systems}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2022},
  doi = {10.1109/taslp.2022.3205753},
}
Figure