Introduction
Non-negative Matrix Factorization (NMF) is a fundamental technique in speech enhancement, where audio signals are decomposed into a dictionary of basis patterns and their corresponding weights. In speaker-independent scenarios, researchers often utilize a large collection of "local dictionaries" to represent diverse human speech sounds. However, selecting the optimal dictionary for a specific noisy audio block remains a significant challenge.
Traditional methods rely on mathematical sparsity penalties (such as the L1 norm) to select patterns, but these often ignore temporal continuity. This neglect can lead to "musical noise" or artifacts caused by erratic switching between dictionary states. This paper proposes a hybrid DNN-HMM framework designed to improve model selection by ensuring both local accuracy and temporal consistency.
Methodology
The proposed system follows a three-stage hybrid approach to enhance noisy speech:
- Clustering & Dictionary Learning: Clean speech is clustered into 20 distinct spectral states using K-means. A local NMF dictionary is then learned for each cluster.
- DNN Classification: A Deep Neural Network (DNN) is trained to predict the probability that a noisy speech block belongs to each of the 20 states, replacing traditional static penalty terms with learned logic.
- HMM & Viterbi Decoding: To prevent rapid "flickering" between states, a first-order Hidden Markov Model (HMM) defines transition probabilities. The Viterbi algorithm identifies the most likely sequence of dictionaries over time, effectively smoothing the transitions.
- Reconstruction: Speech is reconstructed using Wiener-type filtering based on the selected optimal dictionary sequence.
Experimental Setup
- Speech Data: The TIMIT corpus was used, with 20 male speakers for training and 5 different speakers for testing to ensure speaker independence.
- Noise Conditions: 10 non-stationary noise types (e.g., keyboard typing, machine guns) were mixed at 0dB SNR.
- Unsupervised Testing: 100 non-speech environmental sounds were used to test the model's ability to generalize to unseen noise.
Key Insights and Results
The integration of the HMM provides a critical "smoothing" effect. While a DNN-only approach might be confused by sudden bursts of noise, the HMM's transition probabilities act as a temporal anchor, maintaining the correct speech state.
Quantitatively, the DNN-HMM method outperformed several baselines:
- Signal-to-Distortion Ratio (SDR): The proposed method reached 9.07 dB, significantly higher than the Mixture of Local Dictionaries (MLD) baseline of 8.28 dB and the noisy baseline of 0.00 dB.
- Perceptual Quality (PESQ): The system scored 2.38, compared to 2.17 for MLD and 1.90 for Log-MMSE.
- Oracle Performance: An "oracle test" (using true labels) yielded 9.37 dB, suggesting the proposed system is approaching the theoretical performance ceiling for this architecture.
Limitations and Future Work
Despite its success, the model showed a performance drop of approximately 1.4 dB when encountering noise types not present in the training set, indicating sensitivity in the DNN component. Future research may explore "relaxing" the single-dictionary constraint to allow for dictionary blending, or implementing higher-order HMMs to capture longer-term temporal dependencies and speech rhythms.
Summary updated