Stochastic Adaptive Neural Architecture Search for Keyword Spotting

Tom Véniat, Olivier Schwander, Ludovic Denoyer

TL;DR

SANAS (Stochastic Adaptive Neural Architecture Search) is a novel framework that dynamically adjusts a neural network's complexity on-the-fly during inference. By using smaller sub-networks for "easy" inputs (like silence) and larger ones for "hard" inputs (like actual speech), it achieves higher accuracy in keyword spotting while using up to 70% fewer computations than static state-of-the-art baselines.

What Problem Does This Paper Address?

Keyword spotting (KWS)—the technology behind "Hey Siri" or "OK Google"—requires devices to listen continuously to their environment. Traditionally, this is done by sliding a fixed-size neural network over an audio stream.

The challenge is twofold:

  1. Efficiency: Most of the time, the environment is quiet or contains only background noise. Using a massive, power-hungry model to process silence is a waste of battery life on mobile and IoT devices.
  2. Static Rigidity: Standard Neural Architecture Search (NAS) methods find a single "best" architecture that remains the same for every input. However, an audio stream is inherently variable; some segments are much harder to classify than others.

The authors aim to bridge this gap by creating a model that "decides" how much computational effort to expend at every millisecond of the audio stream.

Methods

The authors introduce SANAS, which treats architecture selection as a dynamic, recurrent process.

  • The Super-Network: They start with a large "Super-Network" (based on the popular cnn-trad-fpool3 architecture) and add multiple shortcut connections. This graph contains many possible sub-graphs (architectures) of varying depths and widths.
  • Recurrent Policy: Unlike static NAS, SANAS uses a Recurrent Neural Network (RNN) to maintain a latent state ($z_t$) of the audio context. At each timestep, this RNN generates a distribution over possible architectures.
  • Stochastic Sampling: A specific architecture is sampled from this distribution. If the context suggests silence, the policy learns to sample a "lightweight" sub-graph. If it detects the start of a word, it samples a "heavyweight" sub-graph.
  • Budgeted Learning: The model is trained end-to-end using a loss function that balances two competing goals:
  • Accuracy: Correctness of the keyword detection.
  • Cost: The number of Floating Point Operations (FLOPs) used. A hyperparameter ($\lambda$) allows researchers to tune the trade-off between power saving and performance.

  • Optimization: Because the architecture sampling is non-differentiable, the authors use Monte-Carlo gradient estimation (similar to REINFORCE) with variance reduction to train the policy and the network weights simultaneously.

Datasets

The researchers evaluated SANAS using the Speech Commands dataset, a standard benchmark containing 65,000 one-second clips of 30 different words.

To simulate a real-world "streaming" environment, they:

  • Mixed background noise with randomly located words.
  • Created 1-3 second long audio files.
  • Processed the stream using overlapping 1-second windows every 200ms.
  • Extracted 40 Mel-frequency spectral coefficients (MFCC) as input features.

Key Insights and Ideas

  • Context-Aware Computation: The most striking insight is that the model learns to "pay attention" only when necessary. Figure 5 in the paper shows that as training progresses, the FLOP count for background noise frames drops significantly, while the budget for actual words remains high.
  • Surpassing Static Baselines: SANAS doesn't just save energy; it actually improves performance. By adding shortcut connections and allowing the model to choose its path, the "adaptive" version achieved better accuracy than the static version of the same base model.
  • Word-Specific Budgets: The model naturally learns that some keywords are harder to distinguish than others, automatically allocating more "thinking time" to complex phonemes and less to simpler ones.

Key Results

The results demonstrate a clear win for dynamic architectures:

  • Efficiency Gain: A SANAS configuration achieved a 86.5% match rate using only 37.7M FLOPs per frame.
  • Baseline Comparison: The standard static baseline (cnn-trad-fpool3) achieved a lower 81.7% match rate while requiring 124.6M FLOPs.
  • Extreme Savings: In its most aggressive power-saving mode, SANAS could operate at just 519K FLOPs (a 240x reduction from the baseline) while still maintaining a 61.2% match rate—useful for ultra-low-power "wake-up" modes.

Limitations and Future Work

While the results are promising, the authors note a few areas for growth:

  • Hardware Realities: The paper measures efficiency in FLOPs (theoretical operations). In practice, the overhead of switching architectures on-the-fly on specific hardware (like DSPs or NPUs) needs to be investigated to ensure the theoretical gains translate to actual battery savings.
  • Metric Expansion: Future work could optimize for other constraints beyond FLOPs, such as peak memory footprint or specific latency requirements on edge devices.
  • Scaling: The current "Super-Network" is relatively small. Testing SANAS on much larger, more diverse search spaces could yield even more dramatic efficiency improvements.

Summary updated

Abstract

The problem of keyword spotting i.e. identifying keywords in a real-time audio stream is mainly solved by applying a neural network over successive sliding windows. Due to the difficulty of the task, baseline models are usually large, resulting in a high computational cost and energy consumption level. We propose a new method called SANAS (Stochastic Adaptive Neural Architecture Search) which is able to adapt the architecture of the neural network on-the-fly at inference time such that small architectures will be used when the stream is easy to process (silence, low noise, ...) and bigger networks will be used when the task becomes more difficult. We show that this adaptive model can be learned end-to-end by optimizing a trade-off between the prediction performance and the average computational cost per unit of time. Experiments on the Speech Commands dataset [1] show that this approach leads to a high recognition level while being much faster (and/or energy saving) than classical approaches where the network architecture is static.
BibTeX
@inproceedings{veniat2019stochastic,
  author = {Tom Véniat and Olivier Schwander and Ludovic Denoyer},
  title = {{Stochastic Adaptive Neural Architecture Search for Keyword Spotting}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=AiP0IOlCzt},
}

Loading related content...

Figure