TL;DR
This paper introduces a novel multi-task learning approach for simultaneously detecting and precisely locating wake-up words in audio. By training a network to specifically identify the endpoint of a wake-up word and simultaneously predict its duration, the system achieves significantly improved accuracy in pinpointing both the start and end of spoken commands, outperforming existing methods.
What Problem Does This Paper Address?
Wake-up word detection (like "Hey Siri" or "Okay Google") is crucial for hands-free voice assistants. While current deep learning methods are good at detecting if a wake-up word is present, they struggle with accurately pinpointing its exact start and end times (localization). This imprecision is problematic for downstream tasks like command recognition or speaker diarization, which rely on knowing the precise boundaries of the wake-up phrase. Existing methods often treat detection and localization as separate problems or use techniques that are computationally expensive for embedded devices. This paper aims to solve this by developing a unified, efficient method for simultaneous detection and localization.
Methods
The core of the proposed method is a multi-task learning (MTL) framework built upon a Long Short-Term Memory (LSTM) network. The network is designed with two distinct output sub-layers:
- DEE (Detection & Endpoint Estimation) Task: This sub-layer is trained to identify the endpoint of the wake-up word. Crucially, unlike traditional keyword spotting (KWS) that might label the entire word, this task is trained to "fire" only at the frames immediately surrounding the true endpoint. This focused training is hypothesized to lead to more accurate endpoint detection.
- OE (Onset Estimation) Task: This sub-layer predicts the duration of the wake-up word. Instead of directly regressing the duration (which was found to be unstable), it's framed as a classification problem where different duration ranges are assigned to different output nodes.
The overall loss function is a weighted sum of the losses from these two tasks. The wake-up word's onset is then calculated by subtracting the predicted duration from the detected endpoint. This MTL approach allows the network to leverage shared representations while specializing for each task.
Datasets
- Training Data: 500,000 utterances, comprising 100,000 wake-up words and 400,000 filler utterances. The data was augmented with room impulse responses (RIRs) and various noise types (TV, radio, music) at signal-to-noise ratios (SNRs) between 6-16 dB.
- Test Data: 10,895 far-field utterances, including 895 wake-up word utterances and 10,000 filler utterances. The test set was split into an optimization set (one-third) for tuning offset parameters and an evaluation set (two-thirds).
Key Insights and Ideas
- Endpoint-centric Training is Key: Training the network to specifically detect the endpoint rather than the entire wake-up word segment significantly improves localization accuracy. This is because the endpoint often carries more discriminative acoustic information and is less prone to variability than the onset.
- Duration as Classification: Framing duration prediction as a classification problem over discrete bins, rather than direct regression, leads to more stable and accurate duration estimates.
- Synergy of Tasks: The OE task (duration prediction) acts as a valuable constraint for the DEE task (endpoint detection). The small variation in wake-up word duration helps to refine the endpoint estimation, leading to better overall localization.
- Simultaneous Localization and Detection: The proposed MTL approach effectively integrates detection and localization into a single, efficient model, avoiding the need for separate, potentially complex, post-processing steps.
Key Results
- Onset Estimation Accuracy: The proposed method achieved a relative improvement of 41% for onset estimation accuracy compared to baseline methods (DNN and LSTM) when considering errors within ±5 frames.
- Endpoint Estimation Accuracy: The proposed method achieved a relative improvement of 38% for endpoint estimation accuracy compared to baseline methods when considering errors within ±5 frames.
- Robustness: The proposed method demonstrated high accuracy across various mixing weights (ρ) for the MTL loss, indicating robustness to hyperparameter tuning.
- Keyword Spotting Performance: Receiver Operating Characteristic (ROC) curves showed that the proposed method, especially with a non-zero contribution from the OE task (ρ ≥ 0.5), outperformed baseline methods in terms of False Alarm Rate (FAR) vs. False Rejection Rate (FRR).
Limitations and Future Work
- Offset Parameter Tuning: While the proposed method is less sensitive to the mixing weight (ρ), the offset parameters (∆endpoint, ∆onset) still require optimization, though they are shown to be significantly smaller than for baseline methods.
- Wake-up Word Specificity: The paper focuses on a specific wake-up word ("Néé Yahoo!"). Future work could explore generalization to a wider range of wake-up words and different acoustic conditions.
- Computational Complexity: While designed to be efficient, further exploration into model compression and optimization for extremely resource-constrained devices could be beneficial. The paper mentions that the network size was kept smaller than the DNN baseline to reduce complexity.
Summary updated