Convmixer: Feature Interactive Convolution with Curriculum Learning for Small Footprint and Noisy Far-Field Keyword Spotting

Dianwen Ng, Yunqi Chen, Biao Tian, Qiang Fu, Eng Siong Chng

Abstract: This paper introduces ConvMixer, a highly efficient neural architecture designed for wake-word recognition on resource-constrained edge devices. By replacing heavy attention mechanisms with Multi-Layer Perceptron (MLP) "mixer" units and employing a five-stage curriculum learning strategy, ConvMixer achieves state-of-the-art performance in noisy, far-field environments. With only 119K parameters, the model rivals the accuracy of architectures nearly 50 times its size, providing a robust solution for small-footprint keyword spotting (KWS).

Introduction

Keyword spotting (KWS) is essential for hands-free device activation, yet it faces two primary hurdles: severe hardware constraints on edge devices and environmental noise in far-field scenarios. While large Transformer-based models handle noise well, their computational requirements are prohibitive for deployment on smart gadgets. Conversely, existing lightweight CNNs often fail when the Signal-to-Noise Ratio (SNR) is low. ConvMixer bridges this gap by combining a tiny parameter footprint with high noise robustness.

The ConvMixer Architecture

The core innovation of the model is the ConvMixer block, which facilitates multi-dimensional feature extraction without the overhead of self-attention.

The ConvMixer architecture details the hierarchical blocks and the internal structure of the MLP Mixer unit.

As shown in Figure 1, the architecture utilizes:

  • Multi-Dimensional Feature Extraction: 2D Depthwise Separable (DWS) convolutions capture frequency-domain patterns, while 1D-DWS convolutions handle temporal patterns.
  • The MLP Mixer: Inspired by computer vision architectures, this unit uses simple MLPs to allow information to flow across temporal and frequency channels. This provides a "strong alternative to attention," achieving similar global feature interaction benefits with a significantly lighter memory footprint.

Curriculum-Based Multi-Condition Training

To enhance robustness, the authors employ a curriculum learning strategy. Rather than training on noisy data immediately, the model progresses through five stages of increasing difficulty:

  1. Clean audio only.
  2. Clean + 0dB SNR noise.
  3. Addition of -5dB SNR noise.
  4. Addition of -10dB SNR noise.
  5. Far-field simulation using Room Impulse Response (RIR) data to mimic echoes and distance.

This "tutor-like" approach allows the small model to master basic acoustic features before tackling complex environmental distortions.

Experimental Results

ConvMixer was evaluated on the Google Speech Commands V2 (12-label task) using MUSAN and BUT Speech@FIT databases for noise and reverberation simulation.

Performance comparison of ConvMixer against baseline models across various noise levels and computational metrics.

As detailed in Figure 3, the model demonstrates exceptional efficiency and robustness:

  • Efficiency: With only 119K parameters and 22.2M MACs, it is significantly smaller than competitors like KWT-3 (5.3M parameters).
  • Accuracy: It achieved 98.2% accuracy on the official V2-12 test set.
  • Noise Robustness: In the most challenging -10dB SNR far-field test, the curriculum-trained ConvMixer (denoted as ConvMixer †) achieved 71.88% accuracy, outperforming MatchboxNet (62.35%) and performing comparably to the much larger KWT-3 model.

Ablation studies further confirmed that removing the MLP Mixer caused a ~7% drop in accuracy, highlighting its vital role in cross-channel feature interaction.

Conclusion

ConvMixer demonstrates that massive parameter counts and complex attention mechanisms are not strictly necessary for robust KWS. By leveraging efficient MLP-based mixers and a structured curriculum learning path, the model achieves high performance in noisy far-field conditions while remaining suitable for edge deployment. Future work will explore hardware-specific optimizations and scaling the architecture to larger vocabularies.

Summary updated

BibTeX
@inproceedings{ng2022convmixer,
  author = {Dianwen Ng and Yunqi Chen and Biao Tian and Qiang Fu and Eng Siong Chng},
  title = {{Convmixer: Feature Interactive Convolution with Curriculum Learning for Small Footprint and Noisy Far-Field Keyword Spotting}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2022},
  doi = {10.1109/icassp43922.2022.9747025},
}
Figure