SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement

Robert Rehr, Timo Gerkmann

Deep Neural Networks (DNNs) have significantly advanced speech enhancement, yet they often struggle with a "generalization gap"—failing when encountering noise types or signal levels not seen during training. This paper explores how the choice of input features, network architecture, and training data diversity impacts a model's ability to generalize to real-world, unseen acoustic environments.

The Generalization Gap in Speech Enhancement

While DNN-based models excel in controlled environments, their performance often degrades in the wild. This failure is typically attributed to three factors:

  1. Limited Training Data: A lack of variety in noise samples prevents the model from learning a universal representation of noise.
  2. Architecture Constraints: Simpler networks may fail to capture complex temporal dependencies.
  3. Brittle Features: Standard features, such as log-periodograms, are highly sensitive to the absolute volume (level) of the signal and the specific spectral "texture" of the noise.

The authors investigate which of these factors is most critical for robust performance, proposing a novel feature set to address the "brittle feature" problem.

Proposed Methodology: SNR-NAT

The researchers compare three distinct input feature strategies for training DNNs to predict an Ideal Ratio Mask (IRM):

  • Log-Periodograms: The standard baseline, representing the log-scaled power of the noisy signal.
  • Noise Aware Training (NAT): Appending an estimate of the background noise to the input, providing the network with explicit information about the noise floor.
  • SNR-NAT (Proposed): Instead of raw noise estimates, this method uses statistical estimators to calculate a priori and a posteriori Signal-to-Noise Ratios (SNR). These ratios serve as the primary features.
Comparison of feature extraction pipelines: Periodogram, NAT, and the proposed SNR-NAT.

By using ratios, SNR-NAT features are mathematically independent of the absolute volume of the input signal. This "level independence" ensures that the model's behavior remains consistent whether processing a whisper or a shout.

Experimental Setup and Data Diversity

To evaluate the impact of data diversity, the study utilizes three noise datasets: the small but diverse Hu Noise Corpus, the large but low-diversity CHiME 3, and a new, large, and highly diverse Freesound.org dataset curated by the authors. These features were tested across Feed-Forward and Long Short-Term Memory (LSTM) architectures, with evaluation conducted on completely "unseen" noises to test true generalization.

Key Results

The experimental results highlight that feature choice is often more impactful than architectural complexity.

Performance metrics across varying speech peak levels for different datasets and architectures.

Level Independence and Stability

A critical finding is the stability of SNR-NAT models across different input levels. While baseline models (log-periodogram and NAT) saw significant performance drops at low input levels (e.g., -40 dB), SNR-NAT models maintained near-constant performance. This confirms that SNR-based features effectively decouple the enhancement process from absolute signal volume.

Generalization from Small Data

Even when trained on the tiny Hu dataset, SNR-NAT models demonstrated remarkable robustness to unseen noise. In many cases, a simple feed-forward network using SNR-NAT features outperformed a more complex LSTM using standard features. This suggests that better features can significantly compensate for limited training data.

Quantitative Improvements

Across nearly all test scenarios, SNR-NAT consistently yielded higher scores in POLQA (speech quality) and ESTOI (intelligibility) compared to the baselines. Analysis revealed that SNR-NAT features allow the network to learn a more universal, noise-independent strategy for speech extraction, rather than memorizing specific noise textures.

Conclusion

The study demonstrates that while large, diverse datasets and LSTMs improve performance, the choice of input features is the primary driver of robustness. SNR-NAT features provide a level-independent representation that significantly improves generalization to unseen noise types. Future work will focus on integrating "deep" noise estimators to further refine SNR-NAT and exploring its application in multi-channel microphone array scenarios.

Reference: Rehr, R., & Gerkmann, T. (2021). SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement. arXiv:2004.03512v2.

Summary updated

Abstract

In this paper, we address the generalization of deep neural network (DNN) based speech enhancement to unseen noise conditions for the case that training data is limited in size and diversity. To gain more insights, we analyze the generalization with respect to (1) the size and diversity of the training data, (2) different network architectures, and (3) the chosen features. To address (1), we train networks on the Hu noise corpus (limited size), the CHiME 3 noise corpus (limited diversity) and also propose a large and diverse dataset collected based on freely available sounds. To address (2), we compare a fully-connected feed-forward and a long short-term memory (LSTM) architecture. To address (3), we compare three input features, namely logarithmized noisy periodograms, noise aware training (NAT) and the proposed signal-to-noise ratio (SNR) based noise aware training (SNR-NAT). We confirm that rich training data and improved network architectures help DNNs to generalize. Furthermore, we show via experimental results and an analysis using t-distributed stochastic neighbor embedding (t-SNE) that the proposed SNR-NAT features yield robust and level independent results in unseen noise even with simple network architectures and when trained on only small datasets, which is the key contribution of this paper.
BibTeX
@article{rehr2020snr,
  author = {Robert Rehr and Timo Gerkmann},
  title = {{SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=DPQphP8JpYE},
}
Figure