Efficient data selection for speech recognition based on prior confidence estimation using speech and context independent models

Satoshi Kobashikawa, Taichi Asami, Yoshikazu Yamaguchi, Hirokazu Masataki, Satoshi TakahashiInterspeech 2010

TL;DR

This paper introduces a novel, computationally efficient method for selecting high-quality speech data from massive archives. By estimating "prior confidence" before full speech recognition, it significantly reduces processing costs while achieving comparable data selection performance to traditional methods. This is crucial for large-scale speech analysis in applications like call centers.

What Problem Does This Paper Address?

Analyzing vast amounts of speech data, such as call center recordings, is computationally expensive. While text mining can extract valuable insights, it requires accurate transcriptions. Traditional methods for ensuring transcription accuracy involve confidence measures that rely on the results of a full speech recognition pass. This is inefficient because it means a lot of processing power is spent transcribing data that will ultimately be discarded due to low confidence. The paper addresses this by proposing a way to quickly identify potentially "well-recognized" speech segments before committing to the expensive full recognition process, thereby saving significant computational resources.

Methods

The core innovation is a "prior confidence estimation" technique. Unlike conventional methods that estimate confidence after speech recognition, this approach estimates it beforehand. It achieves this by:

  • Using Speech and Context-Independent Models: Instead of complex triphone models used in full recognition, it leverages simpler, faster monophone (context-independent) and a single "speech" model (a Gaussian Mixture Model trained on all phonemes).
  • Acoustic Likelihood Calculation Only: The estimation process is streamlined to only compute acoustic output likelihoods from these simplified models. It bypasses language model integration and complex state transitions.
  • Frame-Level Confidence: It calculates a confidence score for each speech frame based on the difference in likelihood between the best monophone state and the speech model. This difference reflects how "typical" the acoustic features are. Clearer, more typical speech will have higher confidence.
  • Call-Level Aggregation: Frame-level confidence scores are averaged to produce a single "prior confidence" score for each speech utterance or call.
  • Data Selection: Speech data is then ranked by this prior confidence score, allowing for selective processing of high-confidence segments.

Datasets

  • Training Data: The paper mentions using acoustic features for training the monophone and speech GMMs. The specific dataset size is given as 119.51 hours for male and 104.50 hours for female speakers.
  • Evaluation Data: The evaluation task used a dataset of 19.81 hours of spontaneous Japanese dialog, comprising 17,672 calls from 17 male and 31 female speakers. This data was used for speech recognition and evaluation of the data selection methods.

Key Insights and Ideas

  • Pre-computation is Key: The most significant insight is that a good estimate of recognition quality can be obtained before the full, expensive recognition process. This is achieved by simplifying the acoustic modeling.
  • Acoustic Likelihood as a Proxy for Confidence: The difference in acoustic likelihood between a specific (monophone) model and a general (speech) model serves as an effective proxy for how well a speech segment is likely to be recognized. If the acoustic features strongly match a specific phoneme model (and thus are less ambiguous), confidence is high.
  • Speed vs. Accuracy Trade-off: The proposed method demonstrates that a substantial speedup (over 54x faster confidence estimation) can be achieved with minimal loss in data selection performance compared to methods that require full recognition.
  • Practicality for Big Data: This approach is particularly valuable for scenarios with massive speech archives where computational resources are a bottleneck.

Key Results

  • Data Selection Performance: The proposed "prior" confidence method achieved data selection performance comparable to the conventional "posterior" confidence measure (which uses N-best lists from full recognition) across various selection rates.
  • Speedup: The prior confidence estimation process was over 54 times faster than the conventional posterior confidence estimation.
  • Recognition Rate: By selecting data with high prior confidence, the system improved recognition accuracy compared to random selection ("average") and matched the performance of posterior-based selection.
  • Computational Cost: The total computation time for speech recognition processing was significantly reduced when using the proposed data selection, especially at lower data selection rates.

Limitations and Future Work

  • Language Model Influence: The paper notes that the performance difference between "prior" and "posterior" methods was more pronounced at very low selection rates (around 10%), attributing this to the influence of the language model used in the "posterior" method. This suggests that integrating language model information into the prior estimation could further improve performance.
  • Approximation: The prior confidence is an approximation of state-level posterior probability. While effective, it's not a direct measure of the final recognition outcome.
  • Future Work: The authors imply that further improvements could be made by exploring more sophisticated prior confidence models or by integrating aspects of language models into the prior estimation process. The paper also mentions the use of dual-gender models and a dual-gender selection technique, suggesting ongoing work in adapting models for specific speaker characteristics.

Summary updated

BibTeX
@inproceedings{kobashikawa2010efficient,
  author = {Satoshi Kobashikawa and Taichi Asami and Yoshikazu Yamaguchi and Hirokazu Masataki and Satoshi Takahashi},
  title = {{Efficient data selection for speech recognition based on prior confidence estimation using speech and context independent models}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure