TL;DR
This paper proposes an "active perception" approach for speech recognition, where models trained on clean speech are used to selectively ignore acoustic features that are likely to be corrupted by noise. The core idea is that high-energy speech features are more robust to noise, and by focusing on these, recognition performance can be improved, especially in noisy conditions.
What Problem Does This Paper Address?
Automatic Speech Recognition (ASR) systems trained on clean speech often perform poorly when encountering noisy audio. This performance drop is due to the mismatch between the acoustic features of clean and noisy speech. Traditional methods try to compensate for noise by estimating its characteristics directly from the noisy signal. However, this paper argues that relying solely on data-driven noise estimation might be suboptimal. Instead, they propose leveraging the inherent knowledge within clean speech models to actively decide which acoustic features are most reliable and should be used for recognition, effectively ignoring potentially noisy or irrelevant features.
Methods
The authors introduce a novel approach that modifies the distance calculation within Hidden Markov Models (HMMs). Instead of using all acoustic features, they split the feature vector into two sets: "target" (TR) features, which are considered mandatory for reliable recognition (typically high-energy features from clean speech models), and "non-target" (ZN) features, which are deemed unreliable and are ignored. This selection of TR and ZN features is state-dependent, meaning it can vary for different phonetic states within the HMM.
The core of their method lies in modifying the HMM's probability calculation. The posterior probability of a state $s_j$ given an observation $x$ is redefined to only consider the TR features:
$P(s_j|x) = \frac{P(x_{TR}|s_j)P(s_j)}{P(x_{TR})}$
This means that the decision-making process is guided by the a priori knowledge of which features are expected to have high energy in clean speech. They also describe a decoupling of mixture Gaussian HMMs into single-Gaussian states to facilitate this component selection.
Datasets
The experiments were conducted using the Dutch POLYPHONE corpus, which contains connected digit strings recorded over the public switched telephone network. For noisy speech experiments, babble noise was added to the clean speech at signal-to-noise ratios (SNRs) of 15, 10, and 5 dBA.
Key Insights and Ideas
- Leveraging "A Priori" Knowledge: The paper highlights that clean speech models contain valuable information about which acoustic features are typically important for speech. This knowledge can be actively used to filter out potentially misleading noisy features.
- Focusing on High-Energy Features: The core hypothesis is that spectro-temporal regions with high speech energy are more likely to remain representative even in the presence of noise, making them more robust.
- State-Dependent Feature Selection: The ability to dynamically select features based on the current hypothesized phonetic state is crucial for adapting the recognition process.
- Active Perception: The concept of "active perception" is introduced, where the system actively decides what information to process, rather than passively accepting all available data.
Key Results
- Clean Speech: For clean speech, the authors found that they could discard up to 45% of the model components without negatively impacting recognition performance. In some cases, discarding 30% of components even slightly improved accuracy.
- Noisy Speech: For noisy speech, the performance gains were marginal. Recognition accuracy slightly improved when less than approximately 30% of components were discarded, but performance deteriorated when more than 30% were ignored. For example, at 10 dBA, accuracy improved from 65.7% to 69.8% for models with 864 Gaussians.
Limitations and Future Work
The authors acknowledge that the performance gains for noisy speech were disappointing. They suggest several avenues for future work:
- Feature Enhancement: Applying signal enhancement techniques before or in conjunction with their method might improve the contrast between speech and noise.
- Improved Thresholding: The current method for setting thresholds to discard features might be too simplistic. Future work should explore more sophisticated criteria that combine knowledge of both clean speech distributions and noise characteristics.
- Combining Model-Driven and Data-Driven Approaches: The authors conclude that the most promising direction lies in integrating their model-driven "active perception" approach with data-driven methods, such as feature enhancement or more adaptive thresholding.
Summary updated