A closer look on hierarchical spectro-temporal features (HIST)

Martin Heckmann, Xavier Domont, Frank Joublin, Christian GoerickInterspeech 2008

TL;DR

This paper introduces Hierarchical Spectro-Temporal (HIST) features for speech recognition, inspired by the human auditory cortex. By combining a novel non-linear smoothing preprocessing step with their hierarchical feature extraction, the authors achieve improved robustness in noisy conditions, particularly when fused with existing RASTA-PLP features.

What Problem Does This Paper Address?

Robust speech recognition in noisy environments remains a significant challenge. Traditional speech features often rely solely on spectral information, neglecting the crucial temporal dynamics of speech. While some methods like RASTA-PLP incorporate temporal aspects, this paper aims to further enhance robustness by drawing inspiration from the hierarchical processing observed in the mammalian auditory cortex. The goal is to develop features that better capture the spectro-temporal nature of speech, leading to improved recognition accuracy in the presence of noise.

Methods

The core of this work lies in the Hierarchical Spectro-Temporal (HIST) features, which are built in two hierarchical levels:

  • Level 1: Local Feature Extraction: This stage uses Independent Component Analysis (ICA) to learn a set of receptive fields that capture local spectro-temporal variations in the spectrogram. A Winner-Take-Most (WTM) competition and a non-linear thresholding are applied to select the most salient features.
  • Level 2: Combination Features: In this stage, Non-Negative Sparse Coding (NNSC) is used to learn combination patterns that integrate the local features from the first level. These patterns span the entire frequency range, allowing for the extraction of more complex, global spectro-temporal features.

A key novel contribution is the non-linear smoothing applied to the spectrograms before feature extraction. This smoothing aims to suppress ripples while preserving onsets, a critical aspect for capturing speech dynamics. Additionally, the paper explores a formant enhancement preprocessing step using Mexican Hat filters.

Datasets

  • TIDigits corpus: Used for both isolated and continuous digit recognition tasks.
  • Noisex database: Provided three types of additive noise (Babble, Factory, Car) for evaluation.
  • Aurora-2 framework: The experimental setup for noise addition and HMM training was inspired by this framework.

Key Insights and Ideas

  • Auditory Cortex Inspiration Works: Mimicking the hierarchical processing of the auditory cortex leads to more robust speech features.
  • Non-linear Smoothing is Crucial: The proposed non-linear smoothing method significantly improves performance, especially in noisy conditions, by preserving speech onsets while smoothing out unwanted ripples.
  • HIST Features are Complementary: HIST features provide information that is not captured by traditional MFCC or RASTA-PLP features, leading to performance gains when combined.
  • Hierarchical Structure Matters: The second layer of the HIST architecture, which combines local features into more complex ones, is essential for achieving good performance.

Key Results

  • When combined with RASTA-PLP, HIST features showed significant improvements in word error rates (WER) in noisy conditions compared to RASTA-PLP alone. For instance, in factory noise, the performance of MFCCs degraded rapidly below 15 dB SNR, while HIST+RASTA-PLP maintained better performance.
  • The non-linear smoothing, on average, provided a relative improvement of over 20% compared to linear smoothing across various noise types and SNRs, with an approximate 5% improvement in most cases.
  • In clean conditions, MFCCs performed best, but the differences between MFCCs, RASTA-PLP, and HIST+RASTA-PLP were small and not statistically significant.
  • Tests on the continuous digit recognition task confirmed the general trend observed in isolated digit recognition, suggesting that HIST features scale to more complex tasks.

Limitations and Future Work

  • Computational Cost: The current Matlab implementation is computationally intensive, limiting the extent of evaluation.
  • High SNR Performance: HIST features were found to be inferior to MFCC and RASTA-PLP at high SNR levels. The authors attribute this to significant downsampling after the first hierarchical level and insufficient preprocessing.
  • Babble Noise Anomaly: Babble noise was the only case where the non-linear smoothing did not improve performance over linear smoothing. The authors suggest this might be due to the lack of assumptions on formant locations or numbers in their preprocessing.

Future work could focus on optimizing the computational efficiency of the HIST features and exploring more advanced preprocessing techniques to further improve performance, especially at high SNRs and in challenging noise conditions like babble.

Summary updated

BibTeX
@inproceedings{heckmann2008closer,
  author = {Martin Heckmann and Xavier Domont and Frank Joublin and Christian Goerick},
  title = {{A closer look on hierarchical spectro-temporal features (HIST)}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure