Introduction
Automatic Speech Recognition (ASR) systems frequently encounter performance degradation when deployed in real-world environments characterized by background noise and varying channel conditions. Traditional preprocessing techniques, such as Wiener filtering, often introduce artifacts like "musical noise" that can be more detrimental to ASR accuracy than the original interference. While Non-negative Matrix Factorization (NMF) has emerged as a powerful tool for feature extraction, many existing implementations rely on parallel corpora—clean and noisy versions of the same speech—which are difficult and expensive to acquire. This paper proposes a novel framework that combines NMF with Total Variability Modeling (TVM) to adapt acoustic dictionaries to individual utterances without the need for parallel training data.
Proposed Methodology
The core of the proposed approach is a three-stage pipeline designed to extract robust acoustic features by modeling how spectral dictionaries shift in response to environmental variability.
1. Universal Background Model (UBM)
Rather than training on pristine clean speech, the system utilizes NMF to learn a "universal" dictionary from a large, heterogeneous set of noisy speech recordings. This dictionary captures the broad spectral components present across diverse acoustic environments.
2. Total Variability Modeling
Borrowing from the "i-vector" framework used in speaker verification, the authors assume that any specific utterance's dictionary is a shifted version of the UBM dictionary. They train a total variability matrix ($T$) to learn the subspace of typical dictionary variations caused by noise, speaker characteristics, and channel effects.
3. Utterance-Specific Adaptation
During inference, a low-dimensional "i-vector" is calculated for each new noisy utterance. This vector describes the specific acoustic environment and is used to adapt the UBM dictionary to the current recording. The final acoustic features for the ASR system are the log-activations ($H$) derived from this customized dictionary.
Experimental Evaluation
The method was evaluated using the Aurora 4 corpus (based on Wall Street Journal data) and the DEMAND database of real-world noise (e.g., parks, restaurants). The training set consisted of 7,138 utterances corrupted with six specific noises at 5–15 dB SNR. Testing was conducted on both "seen" noises (included in training) and "unseen" noises (e.g., hallways, stations) to evaluate generalization capabilities.
Results and Discussion
The proposed method was compared against standard baselines, including Mel-Frequency Cepstral Coefficients (MFCC), Power-Normalized Cepstral Coefficients (PNCC), and Convolutive NMF (CNMF).
While the proposed method did not outperform the baselines in terms of raw Word Error Rate (WER) on clean speech, it demonstrated significant stability. A key finding was the smaller performance gap between clean and unseen noise conditions compared to traditional features. Interestingly, performance on "unseen" noise was consistently better than on "seen" noise across all methods; the authors attribute this to the high non-stationarity and background speech present in the "seen" training noises (like restaurant environments), which are inherently more challenging for ASR.
Conclusion and Future Work
The primary contribution of this work is demonstrating that Total Variability Modeling can achieve noise robustness in NMF-based features without requiring parallel training data. The current performance limitations are likely due to the relatively small training set (7k utterances), which may be insufficient to fully define the variability subspace ($T$). Future research will focus on scaling the training to larger datasets like LibriSpeech and incorporating more explicit modeling of channel variability to improve real-world performance.
Summary updated