Introduction
In real-world acoustic environments, speech signals are frequently degraded by two primary factors: reverberation, caused by sound reflecting off surfaces, and additive noise from background sources. Enhancing speech from a single microphone (monaural enhancement) is particularly challenging due to the absence of spatial cues. Traditional methods like Weighted Prediction Error (WPE) excel at dereverberation in clean conditions but falter in noisy environments. Conversely, direct mapping from noisy-reverberant speech to clean speech using deep learning can be overly complex for a single network to learn effectively.
This paper introduces a novel two-stage joint training framework designed to tackle noise and reverberation simultaneously. By leveraging Deep Clustering (DC) to extract robust "deep embedding features" and a supervised Bidirectional Long Short-Term Memory (BLSTM) network for dereverberation, the proposed system achieves significant performance gains over standard baselines.
Proposed Methodology
The architecture consists of two main stages optimized through joint training to ensure the entire pipeline works harmoniously.
- Denoising Stage (Deep Clustering): The first stage employs a Deep Clustering network to map each Time-Frequency (T-F) bin of the input noisy signal into a high-dimensional embedding space. These "noise-free" embedding vectors represent the underlying spectral patterns of the speech and its reverberation, effectively isolating speech components from background noise.
- Dereverberation Stage (Supervised BLSTM): Unlike standard Deep Clustering which uses unsupervised K-means, this framework utilizes a supervised BLSTM network. It takes the noise-free embeddings as input and learns to estimate a mask for the anechoic (clean, non-reverberant) speech.
- Joint Training: The entire pipeline is fine-tuned together. The loss function compares the final estimated magnitude of the anechoic speech against the ground truth. This joint optimization forces the denoising stage to learn embeddings specifically tailored for the subsequent dereverberation task.
Experimental Setup
The model was trained and evaluated using clean utterances from the TIMIT database. Reverberation was simulated using the image-source method with Room Impulse Responses (RIRs) ranging from $RT_{60}$ 0.2s to 2.0s. Noise conditions included:
- Seen Noise: 100 Nonspeech Sounds database used during training.
- Unseen Noise: The NISEX-92 dataset used for testing to evaluate generalization.
- SNR Levels: Training and testing were conducted at -5, 0, 5, and 10 dB Signal-to-Noise Ratios.
Results and Discussion
The proposed method was compared against a WPE baseline and a standard BLSTM-only enhancement system. Performance was measured using Perceptual Evaluation of Speech Quality (PESQ), Cepstral Distance (CD), and Log Likelihood Ratio (LLR).
Impact of Embedding Dimension
The authors investigated the effect of the embedding dimension ($D$). As shown in the results table below, increasing $D$ from 10 to 20 yields noticeable improvements across all metrics. However, further increasing $D$ to 30 or 40 provides diminishing returns while increasing computational complexity. $D=20$ was identified as the optimal balance for performance and efficiency.
Performance Gains and Generalization
The proposed method consistently outperformed both WPE and BLSTM baselines. Notably, the gains were most significant in low SNR conditions. For instance, at -5 dB SNR, the proposed method achieved a PESQ of 2.34, significantly higher than the standard BLSTM (2.17) and WPE (1.55).
Furthermore, the model demonstrated strong generalization capabilities. Even when faced with unseen noise from the NISEX-92 dataset, the deep embedding features remained robust, maintaining a clear performance advantage. Quantitatively, compared to the BLSTM baseline, the proposed method showed a 3.8% relative improvement in PESQ and reduced error metrics (CD and LLR) by 12.3% and 19.1%, respectively.
Conclusion
This research demonstrates that using deep embedding features as a robust intermediate representation significantly improves monaural speech enhancement in noisy and reverberant environments. By jointly training denoising and dereverberation stages, the model learns to optimize for the final target more effectively than disjoint or direct-mapping approaches. Future work will explore extending this framework to multi-channel systems to leverage spatial information alongside the discriminative power of deep embeddings.
Summary updated