Deep Neural Networks (DNNs) have significantly advanced speech enhancement, yet they often struggle with a "generalization gap"—failing when encountering noise types or signal levels not seen during training. This paper explores how the choice of input features, network architecture, and training data diversity impacts a model's ability to generalize to real-world, unseen acoustic environments.
The Generalization Gap in Speech Enhancement
While DNN-based models excel in controlled environments, their performance often degrades in the wild. This failure is typically attributed to three factors:
- Limited Training Data: A lack of variety in noise samples prevents the model from learning a universal representation of noise.
- Architecture Constraints: Simpler networks may fail to capture complex temporal dependencies.
- Brittle Features: Standard features, such as log-periodograms, are highly sensitive to the absolute volume (level) of the signal and the specific spectral "texture" of the noise.
The authors investigate which of these factors is most critical for robust performance, proposing a novel feature set to address the "brittle feature" problem.
Proposed Methodology: SNR-NAT
The researchers compare three distinct input feature strategies for training DNNs to predict an Ideal Ratio Mask (IRM):
- Log-Periodograms: The standard baseline, representing the log-scaled power of the noisy signal.
- Noise Aware Training (NAT): Appending an estimate of the background noise to the input, providing the network with explicit information about the noise floor.
- SNR-NAT (Proposed): Instead of raw noise estimates, this method uses statistical estimators to calculate a priori and a posteriori Signal-to-Noise Ratios (SNR). These ratios serve as the primary features.
By using ratios, SNR-NAT features are mathematically independent of the absolute volume of the input signal. This "level independence" ensures that the model's behavior remains consistent whether processing a whisper or a shout.
Experimental Setup and Data Diversity
To evaluate the impact of data diversity, the study utilizes three noise datasets: the small but diverse Hu Noise Corpus, the large but low-diversity CHiME 3, and a new, large, and highly diverse Freesound.org dataset curated by the authors. These features were tested across Feed-Forward and Long Short-Term Memory (LSTM) architectures, with evaluation conducted on completely "unseen" noises to test true generalization.
Key Results
The experimental results highlight that feature choice is often more impactful than architectural complexity.
Level Independence and Stability
A critical finding is the stability of SNR-NAT models across different input levels. While baseline models (log-periodogram and NAT) saw significant performance drops at low input levels (e.g., -40 dB), SNR-NAT models maintained near-constant performance. This confirms that SNR-based features effectively decouple the enhancement process from absolute signal volume.
Generalization from Small Data
Even when trained on the tiny Hu dataset, SNR-NAT models demonstrated remarkable robustness to unseen noise. In many cases, a simple feed-forward network using SNR-NAT features outperformed a more complex LSTM using standard features. This suggests that better features can significantly compensate for limited training data.
Quantitative Improvements
Across nearly all test scenarios, SNR-NAT consistently yielded higher scores in POLQA (speech quality) and ESTOI (intelligibility) compared to the baselines. Analysis revealed that SNR-NAT features allow the network to learn a more universal, noise-independent strategy for speech extraction, rather than memorizing specific noise textures.
Conclusion
The study demonstrates that while large, diverse datasets and LSTMs improve performance, the choice of input features is the primary driver of robustness. SNR-NAT features provide a level-independent representation that significantly improves generalization to unseen noise types. Future work will focus on integrating "deep" noise estimators to further refine SNR-NAT and exploring its application in multi-channel microphone array scenarios.
Reference: Rehr, R., & Gerkmann, T. (2021). SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement. arXiv:2004.03512v2.
Summary updated