Abstract
Sparse Imputation (SI) is a powerful missing data technique for robust automatic speech recognition (ASR), yet it typically lacks a mechanism to quantify the reliability of its reconstructed features. This paper introduces heuristic uncertainty measures designed to estimate the accuracy of these reconstructions. By integrating these measures into the ASR decoding process via uncertainty decoding, we demonstrate significant improvements in recognition accuracy. Experimental results on the SPEECON database corrupted with babble noise show up to a 15% relative reduction in error rates compared to standard SI.
Introduction
Automatic Speech Recognition (ASR) systems frequently encounter performance degradation in noisy environments. Missing Data Techniques (MDT) address this by identifying "reliable" (speech-dominant) and "unreliable" (noise-dominant) regions of a speech spectrogram. Sparse Imputation (SI) is a state-of-the-art MDT approach that reconstructs unreliable components by representing them as a sparse combination of clean speech exemplars from a dictionary.
However, standard SI algorithms treat all reconstructed features as equally accurate. In practice, reconstruction quality varies significantly. If a poorly reconstructed feature is treated as ground truth by the ASR decoder, it leads to recognition errors. This research addresses the challenge of measuring SI reconstruction uncertainty to improve decoder robustness.
Methods
The authors propose several heuristic measures to estimate the uncertainty ($\hat{\sigma}$) of reconstructed features in the log-spectral domain. These estimates are then mapped to the Mel-frequency cepstral coefficient (MFCC) domain using a learned linear transformation. The key heuristic measures evaluated include:
- M1 (Baseline): Assigns a constant uncertainty to all reconstructed features.
- M4 (Sparsity-based): Uncertainty is proportional to the number of exemplars used in the reconstruction, assuming that complex representations are less reliable.
- M5 (Mask-based): Uncertainty is proportional to the number of unreliable components in the time-window used for reconstruction.
These uncertainties are integrated into the ASR system through Uncertainty Decoding, which modifies state likelihood calculations by adding the estimated uncertainty to the variance of the acoustic models.
Experimental Setup
The evaluation utilized the Finnish SPEECON database for clean speech and NOISEX-92 for babble and pink noise. The task involved large vocabulary continuous speech recognition (LVCSR) on 1118 sentences corrupted at 10 dB and 5 dB Signal-to-Noise Ratios (SNR). A separate set of 500 sentences was used to train the linear mapping between spectral and acoustic uncertainty domains.
Results and Discussion
The experimental results, summarized in the table below, highlight the effectiveness of the proposed heuristics.
On babble noise at 10 dB SNR, the best heuristic (M5) reduced the Letter Error Rate (LER) from 21.6% (SI baseline) to 18.4%, representing a 15% relative error reduction. The success of M4 and M5 suggests that the reliability of a reconstruction is closely tied to the overall difficulty and context of the segment rather than individual spectral components.
While these heuristics provide clear benefits, a significant gap remains between them and "Oracle" measures (OA and OS), where true clean speech is known. For instance, Oracle measures could potentially drop the LER to 11.4% at 10 dB. Furthermore, at a very noisy 5 dB SNR, the heuristics struggled to consistently outperform the SI baseline, likely due to increased errors in the initial mask estimation.
Conclusion
This work demonstrates that incorporating heuristic uncertainty measures into Sparse Imputation significantly enhances ASR performance in noisy conditions. Future research will focus on developing formal probabilistic frameworks that naturally output predictive variances and exploring non-linear mappings, such as neural networks, to better capture complex uncertainty dependencies across domains.
Summary updated