A comparison of multiple methods for rescoring keyword search lists for low resource languages

Victor Soto, Lidia Mangu, Andrew Rosenberg, Julia HirschbergInterspeech 2014

TL;DR

This paper introduces a two-stage machine learning approach to improve keyword search in low-resource languages, achieving significant gains over existing methods. By first reranking speech recognition outputs and then using logistic regression to refine keyword search results, the system demonstrates substantial improvements in detecting spoken words in languages with limited data.

What Problem Does This Paper Address?

Keyword search (KWS) in low-resource languages is a critical but challenging task. These languages often lack extensive transcribed data and linguistic resources, making it difficult for automatic speech recognition (ASR) systems to accurately identify spoken words. Traditional KWS systems struggle with the inherent ambiguity and variability in speech, especially when dealing with limited training data. This paper tackles the problem of improving the accuracy of KWS in these challenging scenarios by developing a cascaded machine learning approach that refines ASR outputs before performing the final keyword search.

Methods

The proposed solution is a two-stage cascaded machine learning approach:

  • Stage 1: Reranking Confusion Networks (CNs):
    • Confusion Networks (CNs) are generated from ASR lattices, providing a compact representation.
  • Arcs within each CN are reranked using a variety of features, aiming to improve ASR performance (e.g., reduce Word Error Rate - TER).
  • Features extracted include lexical, phonetic, syntactic proxy, and probabilistic information at the arc, bin, segment, and conversation levels.
  • Quadratic Programming Feature Selection (QPFS) is used for feature selection.
  • SVM-ranking classifiers are trained for this reranking task.
  • Stage 2: Rescoring Keyword Search Hits:
    • Keyword search hypotheses are generated from the rescored ASR output.
  • Logistic regression classifiers are trained to detect true hits and false alarms.
  • A rich set of features is extracted from posting lists and matched confusion bins, including:
  • CN features: related to the structure of confusion networks.
  • Posterior features: aggregations of original and rescored posterior probabilities.
  • Rescoring features: scores from various existing rescoring techniques (e.g., exponential normalization, PFA normalization).
  • Acoustic features: derived from pitch contours and voice analysis.
  • Duration features: comparing hit durations to keyword-specific and general durations.
  • Instance-based weights derived from the Term Weighted Value (TWV) metric are used.

Datasets

The experiments were performed on four corpora from the IARPA Babel program for low-resource languages:

  • Assamese
  • Bengali
  • Lao
  • Zulu

These corpora consist of conversational and scripted telephone speech, with transcriptions for a fraction of the dataset and pronunciation lexicons. Experiments were conducted using the Limited Language Packs (LLPs), which involve a 10-hour subset of transcribed audio and a reduced pronunciation lexicon for training.

Key Insights and Ideas

  • Cascaded approach is key: Combining ASR output refinement (CN reranking) with targeted keyword search hit rescoring yields superior results compared to single-stage methods.
  • Feature engineering is crucial: The success of the logistic regression classifier in the second stage heavily relies on a diverse and informative set of features, including acoustic, duration, and scores from prior rescoring techniques.
  • TWV metric drives optimization: The paper highlights the challenge of optimizing for the TWV metric due to unbalanced datasets and keyword-dependent penalties, and proposes using keyword-dependent cost penalties inspired by TWV for training.
  • Low-resource languages benefit significantly: The proposed method shows particularly strong improvements for Out-of-Vocabulary (OOV) queries, which are common in low-resource settings.

Key Results

  • The proposed two-stage cascaded machine learning approach consistently outperforms state-of-the-art rescoring techniques.
  • Relative gains range from 5.02% to 36.98% compared to the next best rescoring method across all keyword lists and types.
  • Improvements are especially significant for OOV subsets, reaching up to 42.4% relative improvement.
  • The method also leads to improvements in ASR performance, with 0.7% to 3.0% absolute reductions in TER (Total Error Rate).
  • Among the compared methods, Reranked Posterior Scores (RPost) generally performed well, while Exponential Normalization was the weakest. The ranking was STO < pFA < β-STO.

Limitations and Future Work

  • Data dependency for TWV features: Computing duration and TWV features requires keyword-specific hits on the training data, which might not always be available. Strategies to overcome this drawback are noted as future work.
  • Overlap in keyword lists: To avoid biased results, common keywords were ignored in MTWV calculations between development and evaluation sets.
  • Future directions: The paper suggests exploring further feature engineering, potentially incorporating more advanced morphological analysis for synthetic languages, and investigating alternative optimization strategies for the TWV metric.

Summary updated

BibTeX
@inproceedings{soto2014comparison,
  author = {Victor Soto and Lidia Mangu and Andrew Rosenberg and Julia Hirschberg},
  title = {{A comparison of multiple methods for rescoring keyword search lists for low resource languages}},
  booktitle = {Interspeech},
  year = {2014},
}
Figure