Phone mismatch penalty matrices for two-stage keyword spotting via multi-pass phone recognizer

Chang Woo Han, Shin Jae Kang, Chul Min Lee, Nam Soo KimInterspeech 2010 · OpenReview

Abstract

In this paper, we propose a novel approach to estimate three types of phone mismatch penalty matrices for two-stage keyword spotting. When the output of a phone recognizer is given, text matching with the phone sequences provided by the specified keyword using the proposed phone mismatch penalty matrices is carried out to detect a specific keyword. The penalty matrices, which are estimated from the training data through deliberate error generation, account for substitution, insertion, and deletion errors. In comparative experiments on a Korean continuous speech recognition task, the proposed approach has shown a significant improvement.

Introduction

Keyword spotting (KWS) is a critical technology for voice-activated devices, enabling them to detect specific trigger words within a continuous audio stream. A common implementation is the two-stage approach: the first stage employs a phone recognizer to convert audio into phonetic sequences, and the second stage uses a lexical decoder to match these sequences against a target keyword.

A major challenge in this architecture is the inherent inaccuracy of speech recognition. Traditional systems often rely on fixed penalties, such as the Levenshtein distance, or simple confusion matrices to handle mismatches. However, these methods frequently fail to capture the complex ways phones are misidentified in real-world scenarios, particularly when training data for specific error types is sparse. This paper addresses this gap by proposing a method to estimate Phone Mismatch Penalty (PMP) matrices through deliberate error generation.

Proposed Methodology

The proposed system utilizes a two-stage architecture designed for robustness and flexibility. In the first stage, Mel-frequency cepstral coefficients (MFCCs) and Hidden Markov Models (HMMs) generate N-best phone sequences or phone lattices.

Two-stage keyword spotting architecture

The core innovation lies in the estimation of the penalty matrices ($PM_{sub}$, $PM_{ins}$, and $PM_{del}$). Rather than relying solely on naturally occurring errors, the authors mathematically simulate errors using training data. By comparing the log-likelihood of correct phone sequences against "corrupted" versions (where phones are substituted, inserted, or deleted), the system predicts likely errors even if they are infrequent in the training set. These penalties are then integrated into a dynamic programming string-matching algorithm. Non-negative control parameters ($\alpha_{sub}, \alpha_{ins}, \alpha_{del}$) are introduced to fine-tune the balance between different error types.

Experimental Setup

The study utilized the SiTEC Reading Sentence DB, a large-scale Korean continuous speech database comprising approximately 85 hours of speech from 400 speakers (200 male, 200 female). The dataset was partitioned into training, development (for penalty matrix estimation), and testing sets. The evaluation task involved spotting 1,000 distinct keywords.

Results and Discussion

The proposed method (PMP-PM) was evaluated against Levenshtein distance (PMP-LD) and standard confusion matrices (PMP-CM) using the Figure of Merit (FOM) metric, representing average detection probability.

ROC curves comparing PMP-PM with baseline methods

Experimental results demonstrated that the proposed method achieved an 81.64% FOM when using phone lattices, significantly outperforming PMP-CM (80.01%) and PMP-LD (77.79%). On average, the proposed penalties yielded an 18.48% relative improvement over the Levenshtein method and a 4.31% improvement over the confusion matrix approach. The Receiver Operating Characteristic (ROC) curves confirm that PMP-PM maintains a higher detection probability across various false alarm rates.

Conclusion

By decoupling phone recognition from keyword matching and utilizing proactive error estimation, the proposed approach offers a computationally efficient and vocabulary-independent solution for KWS. Future work may explore context-dependent (triphone) penalty matrices and further investigate robustness in high-noise environments to enhance real-world applicability.

Summary updated

Abstract

In this paper, we propose a novel approach to estimate three types of phone mismatch penalty matrices for two-state keyword spotting. When the output of a phone recognizer is given, text matching with the phone sequences provided by the specified keyword using the proposed phone mismatch penalty matrices is carried out to detect a specific keyword. The penalty matrices which is estimated from the training data through deliberate error generation are accounting for substitution, insertion and deletion errors. In comparative experiments on a Korean continuous speech recognition task, the proposed approach has shown a significant improvement.
BibTeX
@inproceedings{han2010phone,
  author = {Chang Woo Han and Shin Jae Kang and Chul Min Lee and Nam Soo Kim},
  title = {{Phone mismatch penalty matrices for two-stage keyword spotting via multi-pass phone recognizer}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=1EZXiXn6gU},
}

Loading related content...

Figure