TL;DR
This paper introduces a novel "Redundant Convolutional Encoder-Decoder" (R-CED) network for speech enhancement, specifically targeting babble noise. By leveraging Convolutional Neural Networks (CNNs) and a unique "redundant" feature mapping, they achieve comparable or better denoising performance than traditional methods with significantly fewer parameters, making it ideal for embedded systems like hearing aids.
What Problem Does This Paper Address?
Intelligibility of speech is severely degraded by background babble noise, a common issue in everyday environments. While traditional methods attempt to model and subtract noise, they often struggle with complex babble noise, leading to speech distortion and artifacts. Existing neural network approaches, while effective, tend to be computationally expensive and memory-intensive, limiting their deployment on resource-constrained devices like hearing aids. This paper aims to bridge this gap by developing a memory-efficient, high-performance speech enhancement system using CNNs.
Methods
The core of this work is the proposed Redundant Convolutional Encoder-Decoder (R-CED) network. Unlike standard Convolutional Encoder-Decoder (CED) networks that compress features, R-CED's encoder maps features to a higher dimension by creating "redundant" representations, and the decoder then maps these back to the desired clean speech spectrum. This approach avoids the information loss often associated with pooling layers in traditional CEDs. The network exclusively uses 1-dimensional convolutions along the frequency axis, which is more efficient for speech spectrograms than 2D convolutions. They also explore variations like Cascaded R-CED (CR-CED) and the use of skip connections to further improve performance and training efficiency.
Datasets
The experiments were conducted using the TIMIT database for speech and 27 noise clips from Freesound 2 for babble noise. The training set consisted of 4620 utterances, and the testing set comprised 200 utterances. Noise was added to the speech at a 0dB SNR.
Key Insights and Ideas
- CNNs for Speech Enhancement: The paper demonstrates that CNNs, despite being less common in speech enhancement compared to RNNs or FNNs, can achieve excellent results with significantly fewer parameters.
- Redundant Feature Mapping: The R-CED's novel approach of mapping to higher dimensions with redundant representations, rather than just compressing, is key to its success. This allows for richer feature extraction without the information loss of pooling.
- Memory Efficiency: The significantly smaller parameter count of CNNs (especially R-CED) makes them highly suitable for embedded applications where memory is a critical constraint.
- No Pooling Layers: Eliminating pooling layers in the R-CED architecture prevents the loss of spectral information, which is crucial for accurate reconstruction.
- Skip Connections' Impact: Skip connections were found to be more beneficial for the standard CED architecture than for R-CED, suggesting that R-CED's inherent redundancy already mitigates information loss.
Key Results
- Parameter Efficiency: The proposed CNN-based R-CED network had approximately 68 times fewer parameters than a comparable FNN and 12 times fewer than an RNN, while achieving similar or better denoising performance.
- Performance: The R-CED network, especially with skip connections, outperformed both CED and standard FNN/RNN models in terms of Signal to Distortion Ratio (SDR), Short Time Objective Intelligibility (STOI), and Perceptual Evaluation of Speech Quality (PESQ). For instance, R-CED with skip connections achieved the best performance among the tested CNN architectures.
- Training Efficiency: CNNs could be trained from scratch without pre-training, unlike FNNs/RNNs which required multiple pre-training steps and l2 regularization. This resulted in significantly faster training times for CNNs.
- Input Context: Using 8 STFT vectors (approximately 88ms) as input provided optimal performance, with performance degrading when fewer or more vectors were used.
Limitations and Future Work
- Generalization to Unseen Noise: The authors plan to expand the study to generalize the method to different noise levels and unseen noise types.
- Pruning Networks: Further work is suggested to prune the networks to minimize convolution operation counts, further enhancing efficiency.
- Other Domains: The authors believe R-CED could be applicable to other regression problems beyond speech enhancement.
- Subjective Quality: While objective scores are reported, the paper mentions that subjective listening quality was also used to measure performance, but detailed subjective results are not explicitly presented in the main text.
Summary updated