Abstract
Robust automatic speech recognition (ASR) remains a significant challenge due to the acoustic mismatch between clean training data and noisy test environments. This paper, presented at Interspeech 2010, introduces an enhanced Linear Spline Interpolation (LSI) method for Hidden Markov Model (HMM) adaptation. By moving away from signal-level error minimization and instead training adaptation parameters directly within the speech recognizer's framework using a Maximum Likelihood (ML) criterion, the authors achieve substantial improvements in recognition accuracy. Experimental results on the Aurora 2 corpus demonstrate a 28.1% relative reduction in word error rate compared to standard Vector Taylor Series (VTS) adaptation.
The Challenge of Noisy ASR
ASR systems often fail in real-world conditions because their acoustic models are typically trained on clean speech. To bridge this gap, researchers use "adaptation" to adjust clean models to match noisy environments. The core difficulty lies in the fact that the relationship between clean speech, noise, and the resulting noisy signal is highly nonlinear in the log-spectral domain.
Standard methods like Vector Taylor Series (VTS) attempt to simplify this by linearizing the relationship around specific points. While the authors previously proposed Linear Spline Interpolation (LSI) to provide a better piecewise linear approximation, the original LSI parameters were trained "offline" to minimize mean-squared error (MSE) between signals. This signal-level objective does not always translate to optimal speech recognition performance.
Integrated Spline Parameter Training
This paper addresses the objective mismatch by training the LSI splines to maximize the likelihood of the speech itself. The researchers developed an integrated training algorithm based on a Generalized Expectation-Maximization (EM) framework:
- Linear Spline Interpolation (LSI): The relationship between the signal-to-noise ratios (SNR) of clean and noisy speech is modeled using $K$ linear segments, each with its own slope, intercept, and error variance.
- Maximum Likelihood Training: Instead of using stereo data (clean/noisy pairs), the system uses multi-condition training data and the speech recognizer itself to find optimal parameters.
- Generalized EM: In the E-Step, posterior probabilities of speech states are calculated. In the M-Step, the RProp gradient descent algorithm is used to update the spline parameters to maximize training data likelihood.
This approach also implicitly accounts for phase sensitivity. By treating the spline error as a Gaussian variable, the model naturally captures the uncertainty inherent in the mixing process (phase asynchrony) without requiring complex, hand-tuned phase models.
Experimental Results
The proposed "LSI-ML" method was evaluated using the Aurora 2 corpus, a standard benchmark for robust digit recognition across various noise types and SNRs.
As shown in the performance breakdown by SNR, LSI-ML significantly outperforms standard VTS. It is particularly dominant at higher SNRs (10dB and above), reaching 99.05% accuracy at 20dB. While a version of VTS with a hand-tuned phase factor ($\alpha=2.5$) performs slightly better at very low SNRs, LSI-ML maintains the highest overall average.
The comparison across different test sets (Set A: seen noises; Set B: unseen noises; Set C: channel distortion) highlights the robustness of the integrated training. LSI-ML achieved an average accuracy of 91.57%, representing a 20.7% relative reduction in Word Error Rate (WER) over the original stereo-trained LSI and a 28.1% reduction over VTS.
Conclusion
The transition from signal-level proxy objectives to integrated Maximum Likelihood training marks a significant step forward for LSI-based adaptation. By aligning the adaptation process with the recognition goal, the LSI-ML framework provides a tighter fit to the physical reality of speech-noise interaction. Future work will focus on handling non-stationary noise through online estimation and exploring joint training of HMM and spline parameters.
Summary updated