Contrastive Augmentation: An Unsupervised Learning Approach for Keyword Spotting in Speech Technology

Weinan Dai, Yifeng Jiang, Yuanjing Liu, Jinkun Chen, Xin Sun, Jinglei Tao

Abstract

This paper addresses the persistent challenge in Keyword Spotting (KWS), a fundamental component in speech technology, regarding the acquisition of substantial labeled data for training. Given the difficulty in obtaining large quantities of positive samples and the laborious process of collecting new target samples when the keyword changes, we introduce a novel approach combining unsupervised contrastive learning and a unique augmentation-based technique. Our method allows the neural network to train on unlabeled data sets, potentially improving performance in downstream tasks with limited labeled data sets. We also propose that similar high-level feature representations should be employed for speech utterances with the same keyword despite variations in speed or volume. To achieve this, we present a speech augmentation-based unsupervised learning method that utilizes the similarity between the bottleneck layer feature and the audio reconstructing information for auxiliary training. Furthermore, we propose a compressed convolutional architecture to address potential redundancy and non-informative information in KWS tasks, enabling the model to simultaneously learn local features and focus on long-term information. This method achieves strong performance on the Google Speech Commands V2 Dataset. Inspired by recent advancements in sign spotting and spoken term detection, our method underlines the potential of our contrastive learning approach in KWS and the advantages of Query-by-Example Spoken Term Detection strategies. The presented CAB-KWS provide new perspectives in the field of KWS, demonstrating effective ways to reduce data collection efforts and increase the system's robustness.

Introduction

Keyword Spotting (KWS) is the technology enabling voice-activated interfaces like "Hey Siri" or "OK Google." While effective, current KWS systems are notoriously "data-hungry," requiring thousands of labeled examples for every specific word the system is intended to recognize. This dependency creates a significant bottleneck: adding a new keyword typically requires a laborious and expensive labeling process.

Furthermore, standard convolutional neural networks (CNNs) used in speech processing often suffer from redundancy. They treat every window of audio with equal importance, even though a keyword might only occupy a small fraction of the total segment. This often leads to "non-informative" noise drowning out the actual signal. To address these issues, we propose Contrastive Augmentation-Based Keyword Spotting (CAB-KWS), a framework designed to be both data-efficient and better at focusing on relevant audio segments.

Methodology

The CAB-KWS approach is two-pronged, combining a novel architecture with a clever unsupervised training strategy.

1. The Architecture: Compressed Convolution

Instead of a standard CNN, we propose a Compressed Convolutional Layer designed to filter out noise and focus on informative signal:

  • Attention-based Soft-pooling: Rather than simple max-pooling, this layer uses an attention mechanism to weight different frames. It effectively filters the audio, focusing on informative segments and ignoring silence or noise.
  • Transformer Blocks: After compression, the data passes through Transformer layers to capture long-term dependencies, helping the model understand the temporal structure of a spoken word.

2. The Training: Unsupervised Contrastive Learning

The core innovation is how the model is pre-trained on unlabeled data using a contrastive approach:

  • Augmentation: The system takes an unlabeled audio clip and creates "augmented" versions by changing the speed and volume.
  • Contrastive Loss: The model is taught that the original clip and its augmented versions (the "positives") should have nearly identical high-level feature representations, while clips from different audio files (the "negatives") should be distinct.
  • Reconstruction Task: To ensure the model retains essential acoustic information, it performs an auxiliary task of reconstructing the original audio features from a bottleneck layer.
The CAB-KWS architecture and unsupervised contrastive learning pipeline.

Experimental Setup

The study utilizes three primary datasets to validate the approach:

  • Google Speech Commands V2: The gold standard for KWS, used for the final 12-class classification task.
  • Librispeech-100: 100 hours of clean speech used specifically for the unsupervised pre-training phase.
  • HuNonspeech: Used to add realistic background noise (e.g., pink noise, white noise) to the training data to improve robustness.

Key Results and Insights

CAB-KWS demonstrated superior performance across several metrics:

  • Accuracy: The full CAB-KWS model achieved an evaluation accuracy of 88.5% when pre-trained on Librispeech, significantly higher than the Google baseline of 84.7%.
  • Competitive Advantage: It outperformed other popular unsupervised learning methods, including Contrastive Predictive Coding (CPC) and Autoregressive Predictive Coding (APC), by roughly 1-2% in absolute accuracy.
  • Speed vs. Volume: Ablation studies revealed that pre-training the model to be invariant to speed changes provided a significantly higher accuracy boost than volume changes.
  • Redundancy Reduction: By using attention-based soft-pooling, the model successfully ignores non-informative parts of the audio stream, maintaining high performance even at a 10 dB Signal-to-Noise Ratio (SNR).

Conclusion

The CAB-KWS framework demonstrates that unsupervised contrastive learning can significantly reduce the need for labeled data in keyword spotting tasks. By focusing on informative audio segments through attention-based compression and learning invariant features through speed-and-volume augmentation, the model achieves state-of-the-art robustness. Future work will explore more complex augmentations, such as pitch shifting, and further optimize the architecture for deployment on power-constrained edge devices.

Summary updated

Abstract

This paper addresses the persistent challenge in Keyword Spotting (KWS), a fundamental component in speech technology, regarding the acquisition of substantial labeled data for training. Given the difficulty in obtaining large quantities of positive samples and the laborious process of collecting new target samples when the keyword changes, we introduce a novel approach combining unsupervised contrastive learning and a unique augmentation-based technique. Our method allows the neural network to train on unlabeled data sets, potentially improving performance in downstream tasks with limited labeled data sets. We also propose that similar high-level feature representations should be employed for speech utterances with the same keyword despite variations in speed or volume. To achieve this, we present a speech augmentation-based unsupervised learning method that utilizes the similarity between the bottleneck layer feature and the audio reconstructing information for auxiliary training. Furthermore, we propose a compressed convolutional architecture to address potential redundancy and non-informative information in KWS tasks, enabling the model to simultaneously learn local features and focus on long-term information. This method achieves strong performance on the Google Speech Commands V2 Dataset. Inspired by recent advancements in sign spotting and spoken term detection, our method underlines the potential of our contrastive learning approach in KWS and the advantages of Query-by-Example Spoken Term Detection strategies. The presented CAB-KWS provide new perspectives in the field of KWS, demonstrating effective ways to reduce data collection efforts and increase the system's robustness.
BibTeX
@misc{dai2024contrastive,
  author = {Weinan Dai and Yifeng Jiang and Yuanjing Liu and Jinkun Chen and Xin Sun and Jinglei Tao},
  title = {{Contrastive Augmentation: An Unsupervised Learning Approach for Keyword Spotting in Speech Technology}},
  year = {2024},
  eprint = {2409.00356},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.00356},
}
Figure