Exploring joint equalization of spatial-temporal contextual statistics of speech features for robust speech recognition

Hsin-Ju Hsieh, Jeih-weih Hung, Berlin ChenInterspeech 2012

TL;DR

This paper introduces a novel speech recognition technique that enhances robustness to noise by jointly equalizing not just individual speech features, but also their spatial and temporal relationships. By leveraging polynomial fitting for efficient histogram equalization, the proposed method achieves significant improvements in recognition accuracy while maintaining computational efficiency.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems are highly susceptible to performance degradation caused by variations in acoustic conditions, such as background noise and channel distortions. Traditional methods like Cepstral Mean Normalization (CMN) and Cepstral Mean and Variance Normalization (CMVN) can only compensate for linear distortions. While Histogram Equalization (HEQ) has shown promise in handling non-linear distortions by matching feature distributions, existing HEQ methods often suffer from high storage or computational costs and typically treat each feature dimension independently. This paper addresses the limitations of current HEQ techniques by proposing a method that considers the contextual relationships between speech features, both across different feature dimensions (spatial) and across consecutive time frames (temporal), to achieve more robust speech recognition.

Methods

The core of this paper's contribution lies in two main novelties:

  1. Joint Spatial-Temporal Contextual Equalization: Instead of normalizing each feature vector component in isolation, the authors propose a framework that explicitly models and normalizes the relationships between feature components. This is achieved by using simple differencing and averaging operations to create "spatial" (across dimensions within a frame) and "temporal" (across consecutive frames) contextual features. These contextual features, along with the original features, are then subjected to histogram equalization.
  2. Polynomial-Fitting HEQ (PHEQ): To overcome the computational and storage burdens of traditional HEQ methods (like table-lookup or quantile-based approaches), the authors employ a polynomial-fitting scheme. This method efficiently approximates the inverse of the Cumulative Density Function (CDF) of the training speech data using polynomial functions. This allows for faster and more memory-efficient equalization during recognition.

The overall process involves:

  • Calculating spatial and temporal contextual features using differencing and averaging.
  • Using the polynomial-fitting scheme (PHEQ) to approximate the inverse CDF of the training data.
  • Applying HEQ to the original features and their spatial/temporal contexts using the learned polynomial functions.
  • Combining the equalized features to form the final normalized feature vector.

Datasets

  • Aurora-2 database and task: This is the primary dataset used for evaluation. It's a subset of the TI-DIGITS database containing connected digit utterances. The task involves recognizing these utterances under various noise conditions and Signal-to-Noise Ratios (SNRs), including real-world noises and channel distortions.
  • Training: The acoustic models were trained using clean (uncontaminated) training utterances.

Key Insights and Ideas

  • Context is King (for Robustness): The most significant insight is that considering the relationships between speech features (spatial and temporal context) is crucial for improving noise robustness, beyond just normalizing individual features.
  • PHEQ: Efficient HEQ: The polynomial-fitting approach offers a practical solution to the computational and storage challenges of HEQ, making it more viable for real-world ASR systems.
  • Spatial-Temporal Synergy: Jointly equalizing both spatial and temporal contexts (ST-PHEQ) yields superior results compared to normalizing only one aspect, highlighting the complementary nature of these contextual relationships.
  • Integration Potential: Even though the proposed method doesn't outperform the most advanced front-ends (like AFE) on its own, integrating it with existing robust methods (like AFE) can still lead to significant improvements, suggesting its utility as a complementary technique.

Key Results

  • PHEQ vs. Baseline: PHEQ (especially with higher polynomial orders like PHEQ(5) and PHEQ(7)) significantly outperforms the baseline MFCC, CMN, and CMVN methods, achieving average accuracy rates around 81%. It performs comparably to the table-lookup HEQ (THEQ).
  • Contextual Normalization Gains: Further normalizing spatial (S-PHEQ) or temporal (T-PHEQ) contexts provides noticeable improvements over basic PHEQ.
  • Joint ST-PHEQ Superiority: The joint spatial-temporal equalization (ST-PHEQ) achieves the best results among the proposed methods, with average accuracy rates around 83.6%.
  • Integration with AFE: Combining ST-PHEQ with the advanced front-end (AFE) results in a relative word error rate (WER) reduction of approximately 8% compared to using AFE alone.

Limitations and Future Work

  • Limitations:
    • The proposed ST-PHEQ method, while effective, does not solely operate on MFCC feature vectors and corresponding contextual statistics without explicit online noise estimation or reduction processes, which is why it doesn't outperform the highly sophisticated AFE front-end on its own.
  • Future Work:
    • Explore integration with other robustness methods.
  • Conduct larger-scale ASR experiments to further validate the findings.
  • Investigate more elaborate filtering processes that can render spatial and temporal contexts simultaneously.

Summary updated

BibTeX
@inproceedings{hsieh2012exploring,
  author = {Hsin-Ju Hsieh and Jeih-weih Hung and Berlin Chen},
  title = {{Exploring joint equalization of spatial-temporal contextual statistics of speech features for robust speech recognition}},
  booktitle = {Interspeech},
  year = {2012},
}

Loading related content...

Figure