Abstract
Performance of speech recognizers is typically degraded by deleterious properties of the acoustic environment, such as multipath distortion (reverberation) and ambient noise. The degradation becomes more prominent as the microphone is positioned more distant from the speaker, for instance, in a teleconferencing application. Mismatched training and testing conditions, such as frequency response, microphone, signal-to-noise ratio (SNR), and room reverberation, also degrade recognition performance. Among available approaches to handling mismatches between training and testing conditions, a popular one is to retrain the speech recognizer under new environments. Hidden Markov models (HMM) have to date been accepted as an effective classification method for large vocabulary continuous speech recognition, e.g., the ARPA-sponsored SPHINX and DECIPHER. Retraining of HMM-based recognizers is a complex and tedious task. It requires recollection of speech data under corresponding conditions and reestimation of HMM's parameters. Particularly great time and effort are needed to retrain a recognizer which operates in a speaker-independent mode, which is the mode of greatest general interest.
Introduction
Automatic Speech Recognition (ASR) systems have historically struggled with the "distant microphone" problem. While systems like SPHINX and DECIPHER—based on Hidden Markov Models (HMMs)—achieve high accuracy with close-talking headsets, their performance degrades sharply in real-world environments. Factors such as multipath distortion (reverberation) and ambient noise create a significant mismatch between the clean data used for training and the noisy data encountered during testing.
Traditionally, the solution to this mismatch has been to retrain the entire HMM on data specific to the target environment. However, this is a computationally expensive and logistically tedious process, especially for speaker-independent systems. This research presents a hybrid approach that utilizes microphone arrays and neural network mapping to achieve environmental robustness without the need for model retraining.
System Architecture
The proposed system introduces a two-stage "front-end" designed to clean acoustic signals before they reach the primary HMM recognizer. This architecture focuses on adapting the testing environment to the training environment at the feature level.
1. Microphone Array (MA) Processing
The first stage employs a 1-D beamforming line array. Unlike standard omnidirectional microphones, this array provides autodirective sound pickup. By focusing on the speaker's location, the array physically filters out spatial noise and reduces the initial impact of room reverberation.
2. Neural Network (NN) Mapping
The core innovation of this system is the use of Multi-Layer Perceptrons (MLPs) to perform a non-linear mapping of acoustic features. The system utilizes 13 separate MLPs, each dedicated to one of the 13 cepstrum coefficients used by the SPHINX recognizer.
- Temporal Context: Each MLP processes a window of nine frames (the current frame plus four preceding and four succeeding frames). This temporal window allows the network to account for the way reverberation "smears" speech signals over time.
- Feature Transformation: The NNs are trained to transform distorted coefficients from the microphone array into "clean" coefficients that resemble those from a high-quality head-mounted microphone. This allows the back-end SPHINX model to process the data as if it were recorded in ideal conditions.
Experimental Methodology
To evaluate the system, the researchers developed a specialized speech corpus consisting of 80 speakers (50 male, 30 female). The content included 20 command words, 10 digits, and 10 continuous sentences from the Resource Management task.
Recordings were conducted in a 6x6x2.7 meter laboratory with a 0.5-second reverberation time. Three simultaneous audio sources were captured for comparison:
- A head-mounted microphone (the gold standard).
- A 1-D line array.
- A standard desk-mounted PZM microphone.
All signals were sampled at 16 kHz with 16-bit linear quantization.
Results and Discussion
The impact of the hybrid MA/NN system on isolated utterance recognition was substantial. In the testing environment, a standard desk-mounted microphone yielded a baseline accuracy of only 13%. By implementing the combined microphone array and neural network mapping, accuracy rose to 85%.
Crucially, the MA/NN system in an "unmatched" condition (where the mapping was applied to new acoustic data) outperformed a SPHINX model that had been specifically retrained on the array data. This suggests that the neural network's ability to invert environmental distortion is more efficient and effective than attempting to teach an HMM to recognize distorted speech directly.
Conclusion and Future Work
This research demonstrates that a modular neural network front-end can effectively bridge the gap between distant-talking environments and clean-speech recognizers. By treating environmental noise as a transformation problem to be inverted, the system achieves high accuracy without the overhead of HMM retraining.
Future work aims to extend these results to continuous speech recognition and integrate 2-D and 3-D microphone arrays for superior spatial filtering. Additionally, the researchers are working toward a real-time prototype, addressing the computational requirements of running multiple MLPs alongside large-vocabulary recognition engines.
Summary updated