TL;DR
This paper introduces a novel unsupervised method for adapting keyword spotting systems to noisy environments by adjusting feature detector thresholds. This approach significantly outperforms traditional supervised HMM adaptation in white and pink noise, and the point process models themselves show inherent robustness to babble noise.
What Problem Does This Paper Address?
Keyword spotting, the task of identifying specific words within continuous speech, is notoriously challenging in noisy conditions. Traditional methods often rely on frame-based acoustic models (like Hidden Markov Models or HMMs) that process speech in discrete time segments. These models can struggle with the variability and irrelevant information present in noisy speech. The authors propose an alternative approach based on "sparse point process models" (PPMs) that represent speech as temporal patterns of acoustic events, inspired by neuroscience. The core problem this paper tackles is how to make these PPMs robust to noise, specifically by developing an unsupervised adaptation strategy that doesn't require extensive labeled noisy data.
Methods
The paper builds upon their previous work on PPMs for keyword spotting. The core idea is to represent speech not as continuous feature vectors but as sparse temporal "point patterns" generated by feature detectors. These detectors are based on underlying Gaussian Mixture Models (GMMs) trained on clean speech.
The novel contribution is the unsupervised adaptation strategy:
- Threshold Adaptation: Instead of adapting the complex acoustic models themselves (like GMM parameters), the authors propose adapting the thresholds of the feature detectors. The goal is to adjust these thresholds so that the rate of "background" (non-keyword) firings from each detector remains consistent with what was observed in clean speech, even in the presence of noise. This is achieved by finding a new threshold that minimizes the difference between the observed noisy firing rate and the original clean speech firing rate for each feature detector.
- Point Process Model (PPM) Core: The keyword spotting itself uses a PPM framework. It models keyword occurrences with an inhomogeneous Poisson process and background speech with a homogeneous Poisson process. The keyword detector function is based on a log-likelihood ratio comparing these two models.
- Baseline HMM System: For comparison, they use a standard keyword-filler HMM system. Adaptation for the HMM is done using supervised Maximum Likelihood Linear Regression (MLLR) on the GMM means.
Datasets
- TIMIT sx/i training set: Used for training the monophone acoustic models (GMMs) and the clean speech background models.
- TIMIT sal/sx/si test set: Used for evaluating the systems. This set totals 1.3 hours of speech and contains at least 168 instances of each keyword.
- Noise Data: The paper uses noise samples from the NOISEX-92 database for white, pink, and babble noise at various Signal-to-Noise Ratios (SNRs).
- Adaptation Data: For adaptation, both systems were provided with 20 minutes of noisy training speech.
Key Insights and Ideas
- Thresholds as a Robust Adaptation Lever: The most significant insight is that adapting simple detector thresholds can be a surprisingly effective and efficient way to achieve noise robustness, outperforming complex model adaptation in many scenarios.
- PPMs' Inherent Babble Noise Resilience: The sparse nature of PPMs provides an unexpected advantage against non-stationary babble noise. Because the models are based on event occurrences rather than continuous frames, they are less affected by the intermittent corruption of babble noise, as long as the noise doesn't directly coincide with keyword events.
- Unsupervised Advantage: The proposed unsupervised threshold adaptation requires significantly less labeled data and is much faster than supervised MLLR, making it suitable for rapid online adaptation.
- Noise Affects Detection Certainty: Figure 2 visually demonstrates how noise reduces the certainty of feature detectors (e.g., for the [aa] phone), necessitating a lower threshold to maintain the original firing rate.
Key Results
- Superior Performance in White and Pink Noise: The adapted PPM system consistently achieved higher Figure-of-Merit (FOM) scores than the supervised MLLR-adapted HMM system across various levels of white and pink noise. For example, in white noise at 10 dB SNR, the adapted PPM achieved an average FOM of 21.3%, while the adapted HMM achieved 7.0%.
- Babble Noise Robustness: The non-adapted PPM system was significantly more robust to babble noise (at 20-30 dB SNR) than the non-adapted HMM system.
- Rapid Adaptation: PPM adaptation showed performance improvements even with as little as 10 seconds of adaptation data, whereas MLLR adaptation degraded performance when given less than one minute of data.
- MLLR Limitations: Supervised MLLR adaptation showed inconsistent gains and could significantly harm performance, especially in low noise conditions where the unadapted system performed well. Unsupervised MLLR was found to degrade performance.
Limitations and Future Work
- Assumptions: The adaptation strategy relies on two key assumptions: (i) the timing and relative strengths of local maxima in detector functions are preserved, and (ii) noise corrupts each phone detector similarly inside and outside keywords. The paper notes that the observed success validates these assumptions to some degree.
- Future Work: While not explicitly detailed, the success of threshold adaptation suggests further exploration into unsupervised adaptation strategies for sparse models. The inherent robustness to babble noise also opens avenues for investigating how to further leverage this property. The paper also mentions that MLLR can be extended to adapt covariances, but the gains were marginal in their experiments.
Summary updated