On Front-end Gain Invariant Modeling for Wake Word Spotting

Yixin Gao, Noah D. Stein, Chieh-Chi Kao, Yunliang Cai, Ming Sun, Tao Zhang, Shiv Vitaladevuni

Abstract

Wake word (WW) spotting is challenging in far-field due to the complexities and variations in acoustic conditions and the environmental interference in signal transmission. A suite of carefully designed and optimized audio front-end (AFE) algorithms help mitigate these challenges and provide better quality audio signals to the downstream modules such as WW spotter. Since the WW model is trained with the AFE-processed audio data, its performance is sensitive to AFE variations, such as gain changes. In addition, when deploying to new devices, the WW performance is not guaranteed because the AFE is unknown to the WW model. To address these issues, we propose a novel approach to use a new feature called $\Delta$LFBE to decouple the AFE gain variations from the WW model. We modified the neural network architectures to accommodate the delta computation, with the feature extraction module unchanged. We evaluate our WW models using data collected from real household settings and showed the models with the $\Delta$LFBE is robust to AFE gain changes. Specifically, when AFE gain changes up to $\pm$12dB, the baseline CNN model lost up to relative 19.0% in false alarm rate or 34.3% in false reject rate, while the model with $\Delta$LFBE demonstrates no performance loss.

Introduction

Wake word (WW) spotting systems, such as those used for "Alexa," are the primary interface for smart speakers. These systems must operate reliably in noisy, far-field environments characterized by significant acoustic variability. To handle these challenges, an Audio Front-End (AFE) performs tasks like echo cancellation and noise reduction before passing signals to the WW spotter.

Alexa wake word spotting pipeline architecture

However, WW models are often highly sensitive to the specific gain (volume) levels produced by the AFE. Changes in AFE software or hardware can shift the model’s operating point, leading to increased False Alarms or False Rejects. Traditionally, this required costly manual retuning and data collection for every device configuration.

Methodology: Achieving Gain Invariance with $\Delta$LFBE

The researchers propose replacing standard Log Mel-filterbank energy (LFBE) features with $\Delta$LFBE—the temporal difference between consecutive frames—to decouple AFE gain variations from the WW model.

The Math of Invariance

In the logarithmic domain, a constant gain multiplier $c$ results in an additive constant ($2 \log |c|$). By calculating the difference between frames ($\Delta LFBE = LFBE_t - LFBE_{t-1}$), this additive constant is mathematically eliminated, making the feature inherently invariant to volume shifts. This provides a mathematical guarantee of robustness that simple data augmentation cannot match.

Architectural Integration

To maintain compatibility with existing device firmware, the delta operation was "folded" into the neural network itself:

  • CNNs: A frozen first layer acts as a temporal difference filter with weights $[-1, 1]$.
  • DNNs: A "zero-sum constraint" was applied to the first layer's weights, forcing the network to learn differences rather than absolute values.

Evaluation and Results

The models were tested using 8.7 million annotated utterances and simulated gain changes from -12dB to +12dB. To ensure results weren't biased by clipping or quantization, Hard Dynamic Range Compression (HDRC) was used during simulation.

CNN performance comparison across gain levels

Key Findings

  • Baseline Vulnerability: The standard CNN model saw a relative 19% increase in False Alarms or a 34.3% increase in False Rejects when gain shifted by $\pm$12dB.
  • $\Delta$LFBE Resilience: The proposed models showed zero performance loss under the same conditions, with performance curves overlapping perfectly across all gain levels.
  • Hardware Generalization: On "Type B" devices (unseen during training), the $\Delta$LFBE model outperformed the baseline even at nominal gain, suggesting better generalization to new hardware environments.
  • Generalization: In the DCASE audio tagging task, $\Delta$LFBE features maintained a stable F1 score across all gain levels, whereas standard features degraded significantly at high gain (+18dB).

Conclusion

By adopting $\Delta$LFBE features, wake word systems can achieve mathematical gain invariance. This approach eliminates the need for device-specific threshold tuning and ensures robust performance across diverse hardware and software updates. Future research directions include exploring how $\Delta$LFBE interacts with other types of AFE-induced distortions, such as non-linear processing, and testing in more diverse "in-the-wild" use cases.

Summary updated

Abstract

Wake word (WW) spotting is challenging in far-field due to the complexities and variations in acoustic conditions and the environmental interference in signal transmission. A suite of carefully designed and optimized audio front-end (AFE) algorithms help mitigate these challenges and provide better quality audio signals to the downstream modules such as WW spotter. Since the WW model is trained with the AFE-processed audio data, its performance is sensitive to AFE variations, such as gain changes. In addition, when deploying to new devices, the WW performance is not guaranteed because the AFE is unknown to the WW model. To address these issues, we propose a novel approach to use a new feature called $Δ$LFBE to decouple the AFE gain variations from the WW model. We modified the neural network architectures to accommodate the delta computation, with the feature extraction module unchanged. We evaluate our WW models using data collected from real household settings and showed the models with the $Δ$LFBE is robust to AFE gain changes. Specifically, when AFE gain changes up to $\pm$12dB, the baseline CNN model lost up to relative 19.0% in false alarm rate or 34.3% in false reject rate, while the model with $Δ$LFBE demonstrates no performance loss.
BibTeX
@misc{gao2020front,
  author = {Yixin Gao and Noah D. Stein and Chieh-Chi Kao and Yunliang Cai and Ming Sun and Tao Zhang and Shiv Vitaladevuni},
  title = {{On Front-end Gain Invariant Modeling for Wake Word Spotting}},
  year = {2020},
  eprint = {2010.06676},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.06676},
}

Loading related content...

Figure