Discriminative language modeling using simulated ASR errors

Preethi Jyothi, Eric Fosler-Lussier

Abstract

Standard language models (LMs) are typically trained using Maximum Likelihood Estimation (MLE), which minimizes perplexity but does not directly optimize for Word Error Rate (WER). Discriminative training (DT) offers a solution by teaching models to distinguish between correct and confusable incorrect hypotheses. However, traditional DT requires large amounts of paired audio-transcript data to generate error lattices. This paper proposes a framework to simulate ASR errors using only text transcriptions and a phonetic confusion matrix, achieving WER reductions comparable to models trained on real audio data.

Introduction

The primary goal of speech recognition is to minimize the Word Error Rate (WER). While MLE-trained LMs are effective, they lack the ability to explicitly penalize likely recognition errors. Discriminative training addresses this by incorporating "confusable" sentences into the training process. The bottleneck for this approach is the need for extensive audio data to generate these hypotheses. This work introduces a method for discriminatively training LMs without requiring any acoustic training data, utilizing a Weighted Finite State Transducer (WFST) framework to simulate typical speech recognition errors.

Methodology: Simulating Errors with WFSTs

The authors propose a predictive framework that simulates recognition errors using only text transcriptions. The process is modeled as a series of compositions within a WFST.

WFST representation of the n-gram language model transitions used for error simulation.

The core of the simulation is a phonetic confusion matrix ($C$), which models the likelihood of one phone being mistaken for another. This matrix incorporates both empirical data (actual recognizer errors) and acoustic information (mathematical distances between Hidden Markov Models of different phones). By "reversing" a correct transcript into phones, applying the confusion matrix, and mapping back to words, the system generates realistic "confusable" hypotheses.

Learning is performed using the Averaged Perceptron Algorithm. For each training sentence, the model evaluates the $n$-best simulated hypotheses. If the model incorrectly prefers a simulated error over the ground truth, the perceptron updates the n-gram feature weights to penalize the error.

Experimental Setup

The method was evaluated on two primary datasets:

  • Switchboard-1: A corpus of spontaneous telephone speech. A subset of 20,000 sentences was used for training.
  • Fisher Corpus: 10,000 transcript-only sentences were used to test the "audio-free" claim in a cross-domain scenario, evaluating on Switchboard.

Results and Discussion

The experimental results demonstrate that simulated errors are a highly effective surrogate for real audio lattices. On the Switchboard task, the baseline MLE trigram model achieved a 49.7% WER. The discriminative model using simulated errors (FST-Words+HMM) reduced this to 48.8%.

Comparison of Word Error Rate (WER) between baseline, audio-based discriminative training, and the proposed audio-free simulation method.

Crucially, as shown in the results table, the performance of the audio-free simulation ($n=1000$, 48.5% WER) is nearly identical to the model trained on real audio lattices (48.4% WER). This confirms that the patterns of ASR errors are consistent enough to be modeled synthetically. Furthermore, the cross-corpus experiment using Fisher text also yielded a significant improvement (48.8% WER), proving the model's utility when audio for a specific domain is unavailable.

Conclusion

This research proves that acoustic data is optional for effective discriminative language modeling. By capturing the underlying "acoustic confusability" of phones through HMM distances, the simulation framework provides a computationally efficient alternative to generating real word lattices. Future work may involve expanding the feature set beyond simple n-grams to include linguistic features like Part-of-Speech tags and exploring different distributions for selecting $n$-best hypotheses.

Summary updated

BibTeX
@inproceedings{jyothi2010discriminative,
  author = {Preethi Jyothi and Eric Fosler-Lussier},
  title = {{Discriminative language modeling using simulated ASR errors}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-344},
}
Figure