Abstract
In this paper, we propose a novel approach to estimate three types of phone mismatch penalty matrices for two-stage keyword spotting. When the output of a phone recognizer is given, text matching with the phone sequences provided by the specified keyword using the proposed phone mismatch penalty matrices is carried out to detect a specific keyword. The penalty matrices, which are estimated from the training data through deliberate error generation, account for substitution, insertion, and deletion errors. In comparative experiments on a Korean continuous speech recognition task, the proposed approach has shown a significant improvement.
Introduction
Keyword spotting (KWS) is a critical technology for voice-activated devices, enabling them to detect specific trigger words within a continuous audio stream. A common implementation is the two-stage approach: the first stage employs a phone recognizer to convert audio into phonetic sequences, and the second stage uses a lexical decoder to match these sequences against a target keyword.
A major challenge in this architecture is the inherent inaccuracy of speech recognition. Traditional systems often rely on fixed penalties, such as the Levenshtein distance, or simple confusion matrices to handle mismatches. However, these methods frequently fail to capture the complex ways phones are misidentified in real-world scenarios, particularly when training data for specific error types is sparse. This paper addresses this gap by proposing a method to estimate Phone Mismatch Penalty (PMP) matrices through deliberate error generation.
Proposed Methodology
The proposed system utilizes a two-stage architecture designed for robustness and flexibility. In the first stage, Mel-frequency cepstral coefficients (MFCCs) and Hidden Markov Models (HMMs) generate N-best phone sequences or phone lattices.
The core innovation lies in the estimation of the penalty matrices ($PM_{sub}$, $PM_{ins}$, and $PM_{del}$). Rather than relying solely on naturally occurring errors, the authors mathematically simulate errors using training data. By comparing the log-likelihood of correct phone sequences against "corrupted" versions (where phones are substituted, inserted, or deleted), the system predicts likely errors even if they are infrequent in the training set. These penalties are then integrated into a dynamic programming string-matching algorithm. Non-negative control parameters ($\alpha_{sub}, \alpha_{ins}, \alpha_{del}$) are introduced to fine-tune the balance between different error types.
Experimental Setup
The study utilized the SiTEC Reading Sentence DB, a large-scale Korean continuous speech database comprising approximately 85 hours of speech from 400 speakers (200 male, 200 female). The dataset was partitioned into training, development (for penalty matrix estimation), and testing sets. The evaluation task involved spotting 1,000 distinct keywords.
Results and Discussion
The proposed method (PMP-PM) was evaluated against Levenshtein distance (PMP-LD) and standard confusion matrices (PMP-CM) using the Figure of Merit (FOM) metric, representing average detection probability.
Experimental results demonstrated that the proposed method achieved an 81.64% FOM when using phone lattices, significantly outperforming PMP-CM (80.01%) and PMP-LD (77.79%). On average, the proposed penalties yielded an 18.48% relative improvement over the Levenshtein method and a 4.31% improvement over the confusion matrix approach. The Receiver Operating Characteristic (ROC) curves confirm that PMP-PM maintains a higher detection probability across various false alarm rates.
Conclusion
By decoupling phone recognition from keyword matching and utilizing proactive error estimation, the proposed approach offers a computationally efficient and vocabulary-independent solution for KWS. Future work may explore context-dependent (triphone) penalty matrices and further investigate robustness in high-noise environments to enhance real-world applicability.
Summary updated