An auditory system-based feature for robust speech recognition

Qi Li, Frank K. Soong, Olivier Siohan2001 · DOI

TL;DR

This paper introduces a novel speech feature extraction method inspired by the human auditory system, aiming to improve Automatic Speech Recognition (ASR) robustness in noisy environments. By mimicking the outer-middle ear's transfer function and the cochlea's frequency filtering, their proposed feature significantly reduces error rates compared to standard methods like MFCC and LPCC, especially under challenging acoustic conditions.

What Problem Does This Paper Address?

Traditional speech recognition systems often struggle in real-world scenarios due to background noise, reverberation, and variations in recording equipment (e.g., different microphones, handset vs. hands-free). Features like LPCC (Linear Predictive Coding Coefficients) and MFCC (Mel-Frequency Cepstral Coefficients), while effective in clean speech, degrade significantly in adverse conditions. The authors posit that human hearing is remarkably resilient to noise, and by modeling its mechanisms, they can develop more robust ASR features. They aim to bridge the gap between current ASR features and the perceptual robustness of human auditory processing.

Methods

The proposed auditory feature extraction process involves several stages, mimicking human auditory processing:

  1. Outer-Middle-Ear Transfer Function: Instead of a standard preemphasis filter, the system incorporates a transfer function that models the combined acoustic response of the outer and middle ear. This function is designed to have a bandpass characteristic, particularly a high-pass nature in the 0-4 KHz range.
  2. Frequency Conversion to Bark Scale: The linear frequency scale from the Fast Fourier Transform (FFT) is converted to the Bark scale, which better represents how humans perceive frequency. This involves a specific interpolation formula to map linear frequencies to Bark scale points.
  3. Auditory Filtering: A bank of digital auditory filters is designed. These filters are inspired by the shape of cochlear filters (as studied by Patterson) and are spaced equally along the Bark scale. Unlike triangular or PLP filter shapes, these are designed to be roughly symmetric and have a width that smooths out noise while preserving frequency resolution. The shape of these filters can be adjusted during training for optimal performance.
  4. Nonlinearity: A logarithmic nonlinearity is applied to the filtered spectrum, reflecting the non-linear response of the basilar membrane in the cochlea.
  5. Discrete Cosine Transform (DCT): A DCT is applied to convert the logarithmic spectrum into cepstral coefficients. This step also helps smooth out pitch harmonics.
  6. Short-Term Energy: The short-term energy of the speech signal is also included as part of the feature vector.

The overall process is visualized in a schematic diagram (Figure 3 in the paper).

Datasets

The paper evaluates the proposed feature on several datasets:

  • Connected-Digit Recognition:
    • Two CDMA wireless databases: "Handset" and "Lapel," collected in a moving car. These were used for training and testing under noisy conditions.
  • A benchmark dataset for HBT digit models, comprising 21 databases with a total of 44,123 utterances, used for training and testing with both clean and noisy data.
  • Wall Street Journal (WSJ) Recognition Task:
    • The SI-84 training corpus (7,200 utterances) and the WSJ (Nov, 1992)-5K closed vocabulary test set (330 utterances).
  • Noise was added to the original WSJ data to create noisy test sets at various Signal-to-Noise Ratios (SNRs).

The paper also mentions using additive car and babble noise, as well as NYSE noise, to simulate adverse acoustic environments.

Key Insights and Ideas

  • Mimicking Human Hearing Pays Off: The core insight is that by closely modeling the early stages of human auditory processing (outer/middle ear and cochlear filtering), ASR systems can achieve significantly better robustness to noise than features based solely on speech production models or simpler perceptual approximations.
  • Bark Scale is Crucial: The conversion to the Bark scale is highlighted as important for aligning frequency representation with human perception, leading to better performance.
  • Auditory Filter Shape Matters: The proposed auditory filter shapes, which are symmetric and have adjustable widths, are shown to be more effective than those used in MFCC or PLP for noise reduction. Their ability to smooth noise within their bandwidth without losing too much frequency information is key.
  • Flexibility for Optimization: The symmetric nature of the proposed auditory filter allows for minimal parameter adjustment during HMM training, making it adaptable to specific acoustic conditions without excessive complexity.

Key Results

The proposed auditory feature demonstrated significant improvements across various tasks and conditions:

  • Connected-Digit Recognition:
    • Achieved an average of 20% to 30% string error rate (SER) reduction compared to LPCC, MFCC, MEL-LPCC, and PLP features on the noisy connected-digit task.
  • Outperformed other features by 11% to 37% SER reductions on noisy wireless data.
  • Wall Street Journal (WSJ) Recognition:
    • Achieved an average of 8% to 14% word error rate (WER) reduction on the WSJ task in various additive noise conditions.
  • For clean speech, it performed comparably to MFCC, but showed better performance in noise, with WER reductions of 7.6% to 14.3% compared to MFCC at different SNRs.
  • Noise Robustness: The feature showed substantial improvements in SER reduction (up to 34.4% for car noise and 36.0% for NYSE noise) compared to LPCC at low SNRs (10-20 dB).

Limitations and Future Work

  • Computational Complexity: While the paper states the proposed feature used 1% of real time compared to 0.7%-0.8% on others, implying it's comparable or slightly higher, it's still a factor to consider for real-time applications. The authors mention the difference is negligible compared to decoding complexity.
  • Parameter Tuning: Although the filter shape adjustment is minimal, it still requires some tuning during HMM training to achieve optimal performance.
  • Future Work: The authors suggest that the ASR performance is "partially improved" by the outer-middle-ear transfer function, with the "major improvement" coming from the new auditory filters. This implies further exploration into optimizing the outer-middle-ear modeling could yield even greater gains. They also mention the flexibility of the filter shape adjustment as a direction for future optimization.

Summary updated

Abstract

An auditory feature extraction algorithm for robust speech recognition in adverse acoustic environments is presented.The feature computation is comprised of an outer-middle-ear transfer function, FFT, frequency conversion from linear to the Bark scale, auditory filtering, nonlinearity, and discrete cosine transform.The feature is evaluated in two tasks: connected-digit recognition and large vocabulary continuous speech recognition.The tested data were under various noise conditions, including handset and hands-free speech data in landline and wireless communications with additive car and babble noise.Compared with the LPCC, MFCC, MEL-LPCC, and PLP features, the proposed feature has an average 20% to 30% string error rate reduction on the connected-digit task, and 8% to 14% word error rate reduction on the Wall Street Journal task in various additive noise conditions.
BibTeX
@misc{li2001auditory,
  author = {Qi Li and Frank K. Soong and Olivier Siohan},
  title = {{An auditory system-based feature for robust speech recognition}},
  year = {2001},
  doi = {10.21437/eurospeech.2001-164},
}
Figure