To Wake-Up or Not to Wake-Up: Reducing Keyword False Alarm by Successive Refinement

Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Ching Hua Lee, Chouchang Yang, Yilin Shen, Hongxia Jin

Introduction

Keyword Spotting (KWS) systems are integral to modern voice-activated devices, enabling "always-on" listening for trigger phrases like "Hey Siri" or "Alexa." However, these systems frequently suffer from False Alarms (FA)—instances where the device triggers in response to non-keyword audio. High FA rates degrade user experience, deplete battery life on mobile hardware, and raise privacy concerns due to unintended cloud uploads.

Traditional deep learning approaches for KWS typically treat the task as a flat classification problem, attempting to distinguish keywords from all other audio in a single step. Researchers at Samsung Research America have introduced Successive Refinement (SR), a hierarchical approach that leverages the natural structure of audio data to significantly reduce FAs.

Methodology: Successive Refinement

The SR technique decomposes the keyword detection process into a three-stage hierarchy based on the law of total probability. Instead of a monolithic classifier, the system utilizes a shared backbone model followed by three specialized branches:

  1. Speech Branch: Determines if the input is human speech or background noise ($p(S=1|x)$).
  2. Keyword-like Branch: If speech is detected, it evaluates if the utterance resembles a keyword ($p(K=1|S=1, x)$).
  3. Keyword Classification Branch: If the audio is keyword-like, it identifies the specific keyword ($p(c_n|K=1, S=1, x)$).
Comparison between existing flat classification models and the proposed Successive Refinement (SR) architecture.

This "plug-and-play" architecture allows for the reuse of internal representations across branches. A key innovation is data pooling: the speech branch is trained on all speech data (both keywords and non-keywords), providing a much larger and more robust training set than traditional methods that only see keywords as a single class.

Experimental Evaluation

The researchers evaluated SR across several model architectures, including BC-ResNet and Keyword Transformers, using a variety of high-standard datasets:

  • Google Speech Commands V2 (GSC-V2): The primary benchmark for KWS.
  • Childspeech & AVSpeech: Sources for diverse non-keyword speech samples.
  • Audioset & MUSAN: Used for non-speech noise and Out-Of-Domain (OOD) testing to evaluate real-world robustness.

Results and Insights

The SR method achieved up to an 8x reduction in false alarms on in-domain data and a 7x reduction on OOD data. For instance, in a BCR-2 model, the FA rate dropped from 4.23% to 1.11% while maintaining overall accuracy and incurring negligible computational overhead (MACs).

The success of SR stems from forcing the model to learn a hierarchical representation. By separating the detection of speech from the classification of keywords, the model develops more specialized embeddings.

t-SNE visualization of the speech branch embeddings showing the separation of speech from non-speech.

As shown in the t-SNE analysis, the speech branch effectively clusters speech separately from non-speech audio, preventing noise from being misclassified as a keyword early in the pipeline. This hierarchical view prevents the class imbalance issues that often plague standard KWS training.

Conclusion

Successive Refinement offers a computationally efficient way to enhance KWS reliability. By replacing flat classification with a hierarchical refinement process, the method achieves dramatic reductions in false alarms across various architectures, making it highly suitable for real-world, privacy-sensitive, and power-constrained deployments. Future work may involve integrating post-utterance context to further refine detection accuracy.

Summary updated

BibTeX
@inproceedings{saidutta2023wake,
  author = {Yashas Malur Saidutta and Rakshith Sharma Srinivasa and Ching Hua Lee and Chouchang Yang and Yilin Shen and Hongxia Jin},
  title = {{To Wake-Up or Not to Wake-Up: Reducing Keyword False Alarm by Successive Refinement}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095428},
}

Loading related content...

Figure