Optimizing Voice Activity Detection for Noisy Conditions

Ruixi Lin, Charles Costello, Charles Jankowski, Vishwas MruthyunjayaInterspeech 2019 · OpenReview

Abstract

In this work, we focus our attention on how to improve Voice Activity Detection (VAD) in noisy conditions. We propose a Convolutional Neural Network (CNN) based model, as well as a Denoising Autoencoder (DAE), and experiment against acoustic features and their delta features in noise levels ranging from SNR 35 dB to 0 dB. The experiments compare and find the best model configuration for robust performance in noisy conditions. We observe that combining more expressive audio features with the use of DAEs improve accuracy, especially as noise increases. At 0 dB, the proposed model trained with the best feature set could achieve a lab test accuracy of 93.2% (averaged across all noise levels) and 88.6% inference accuracy on device. We also compress the neural network and deploy the inference model that is optimized for the app so that the average on-device CPU usage is reduced to 14% from 37%.

Introduction

Voice Activity Detection (VAD) serves as the critical "gatekeeper" in speech processing systems, determining when a user is speaking and when the system should begin processing input. While VAD performs reliably in quiet environments, real-world scenarios—such as crowded exhibitions or noisy streets—present significant challenges. Distinguishing speech from background "clutter" becomes particularly difficult when noise levels approach the speech level (0 dB Signal-to-Noise Ratio).

Beyond accuracy, practical VAD systems must be computationally efficient to run on mobile devices without excessive battery drain or latency. This research addresses both noise robustness and on-device efficiency by proposing a novel neural architecture and optimization framework.

Methodology

The proposed system utilizes a two-stage neural pipeline designed for robustness:

  1. Denoising Autoencoder (DAE): A 2-layer bottleneck network serves as a pre-processor. It is trained to map noisy audio features to their clean counterparts, effectively filtering the signal before classification.
  2. Convolutional Neural Network (CNN): A CNN architecture analyzes a 21-frame window of audio. CNNs are particularly effective at capturing temporal patterns and reducing input dimensionality compared to standard dense networks.
Table 1: Architecture and dimensions of the proposed CNN model.

The study evaluates various acoustic features, including Mel-Frequency Cepstral Coefficients (MFCCs) and Log Mel-filterbanks (FBank), alongside their dynamic "delta" and "delta-delta" components which capture temporal changes in the signal.

Experimental Setup

The models were trained and evaluated using the following datasets:

  • AISHELL-1: A comprehensive Chinese Mandarin speech corpus for primary training.
  • MWC Noise: Real-world background noise recorded at the Mobile World Congress was added to clean speech to create test sets with SNRs ranging from 35 dB to 0 dB.
  • AURORA2: An English-language benchmark used for cross-dataset validation and comparison with state-of-the-art models.

Results and Analysis

The experiments revealed several key insights into VAD performance in high-noise environments:

  • Feature Engineering: Incorporating delta and delta-delta features significantly improved performance in noisy conditions, highlighting the importance of temporal dynamics.
  • DAE Effectiveness: The Denoising Autoencoder provided the most substantial gains at low SNRs (0 dB and 5 dB), acting as a protective layer for the subsequent classifier.
  • Optimization and Deployment: For on-device execution, the Snapdragon Neural Processing Engine (SNPE) proved superior to standard TensorFlow Mobile, leveraging specialized DSP hardware.
Table 2: Accuracy (%) of MFCC-based models across various Signal-to-Noise Ratios (SNR).

As shown in Table 2, the combined model (MFCC + DAE + Deltas) achieved a remarkable 87.9% accuracy at 0 dB SNR. Overall, the model reached a lab test accuracy of 93.2% (averaged) and maintained 88.6% on physical devices. Furthermore, model compression and SNPE optimization reduced CPU usage on a Snapdragon 820 from 37% to 14%.

Conclusion

This research demonstrates that combining DAEs with CNNs and optimized feature sets can significantly enhance VAD robustness in extreme noise. While the DAE was highly effective for MFCCs, future work will explore deeper architectures for higher-dimensional features like filterbanks. Additionally, the team plans to investigate performance against unseen noise types and further quantization to drive down power consumption for "always-on" mobile applications.

Summary updated

Abstract

In this work, we focus our attention on how to improve Voice Activity Detection (VAD) in noisy conditions. We propose a Convolutional Neural Network (CNN) based model, as well as a Denoising Autoencoder (DAE), and experiment against acoustic features and their delta features in noise levels ranging from SNR 35 dB to 0 dB. The experiments compare and find the best model configuration for robust performance in noisy conditions. We observe that combining more expressive audio features with the use of DAEs improve accuracy, especially as noise increases. At 0 dB, the proposed model trained with the best feature set could achieve a lab test accuracy of 93.2% (averaged across all noise levels) and 88.6% inference accuracy on device. We also compress the neural network and deploy the inference model that is optimized for the app so that the average on-device CPU usage is reduced to 14% from 37%.
BibTeX
@inproceedings{lin2019optimizing,
  author = {Ruixi Lin and Charles Costello and Charles Jankowski and Vishwas Mruthyunjaya},
  title = {{Optimizing Voice Activity Detection for Noisy Conditions}},
  booktitle = {Interspeech},
  year = {2019},
  url = {https://openreview.net/forum?id=Kiivp5DqBy},
}
Figure