Dereverberation based on wavelet packet filtering for robust automatic speech recognition

Randy Gomez, Tatsuya KawaharaInterspeech 2012

TL;DR

This paper introduces a novel method for robust automatic speech recognition (ASR) in reverberant environments by using Wavelet Packet Tree (WPT) decomposition to precisely separate speech from late reflections. By optimizing the WPT structure based on acoustic model likelihood, they achieve more accurate Wiener gain estimation and significantly improve ASR performance compared to existing dereverberation techniques.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems struggle in environments with echoes and reverberation. These reflections smear the original speech signal, causing a mismatch between the acoustic models trained on clean speech and the actual reverberant input. This mismatch degrades ASR accuracy. While previous methods have attempted to mitigate reverberation, they often lack the fine-grained resolution needed to effectively distinguish between the desired speech signal and detrimental late reflections, especially when these subspaces overlap. This paper aims to bridge this gap by proposing a more precise signal analysis technique for dereverberation that is tightly integrated with the ASR process.

Methods

The core of the proposed method lies in Wavelet Packet Tree (WPT) decomposition. Unlike standard wavelet transforms, WPT offers a more flexible way to decompose a signal into frequency subbands, allowing for finer control over the time-frequency resolution. The authors leverage this by:

  • Optimized WPT Selection: Instead of using a fixed WPT structure, they propose a method to select the optimal WPT structure (specifically, the wavelet basis and tree depth) for speech and late reflections separately. This selection is guided by an acoustic model likelihood criterion, meaning the WPT structure is chosen to best represent the characteristics that are important for the ASR system. This is a key innovation, linking the dereverberation process directly to the ASR's needs.
  • Entropy-based Criteria: To manage the vast number of possible WPT structures, they employ entropy-based criteria to guide the tree splitting process, ensuring that the decomposition focuses on informative signal components.
  • Wiener Filtering: Once the optimal WPTs for speech and late reflection are identified, the reverberant signal is decomposed using these structures. A Wiener gain is then calculated within each wavelet packet coefficient to suppress the estimated late reflection component. This gain is applied framewise to the reverberant signal's WPT coefficients.
  • Acoustic Model Integration: The selection of the WPT is crucial. For speech, the WPT is trained using a Gaussian Mixture Model (GMM) of clean speech. For late reflections, synthetic reverberant signals are generated, and the WPT is selected to maximize the likelihood of these synthetic late reflections under a dedicated model.

Datasets

  • Training: The Japanese Newspaper Article Sentence (JNAS) corpus, approximately 60 hours of speech, was used for training.
  • Testing: A test set of 200 sentences uttered by 50 speakers was used.
  • Synthetic Data: Synthetic reverberant data was produced by convolving clean speech with automatically generated Room Impulse Responses (RIRs) for evaluation against mismatch conditions. Actual reverberant data with known reverberation times (T60) of 200ms, 400ms, and 600ms was used for the main evaluation.

Key Insights and Ideas

  • WPT for Fine-grained Subspace Discrimination: The flexibility of WPT decomposition is crucial for precisely separating the spectral characteristics of speech from late reflections, a task that simpler filterbanks struggle with.
  • Acoustic Model Likelihood as a Guide: Directly linking the WPT selection process to the ASR acoustic model's performance is a powerful idea. This ensures that the dereverberation focuses on what's important for recognition, rather than just general signal enhancement.
  • Separate Optimization for Speech and Reflections: Optimizing WPT structures independently for speech and late reflections, rather than using a single generic structure, leads to more effective discrimination and better Wiener gain estimation.
  • Integration is Key: The paper highlights that successful dereverberation for ASR requires tight integration between the signal processing (WPT filtering) and the recognition model itself.

Key Results

The proposed method achieved state-of-the-art results in word accuracy for a Large Vocabulary Continuous Speech Recognition (LVCSR) task.

  • In matched conditions (test reverberation matches training), the proposed method (I) achieved 71.5% word accuracy for a T60 of 600ms. This significantly outperforms other methods, including a baseline WPT filtering approach (H) which achieved 63.7%, and multi-band spectral subtraction (D) at 61.6%.
  • Even in mismatched conditions (where the system's assumed T60 differs from the actual T60), the proposed method demonstrated superior robustness, outperforming previous wavelet filtering techniques.

Limitations and Future Work

  • Computational Cost: While WPT offers finer resolution, the decomposition and optimization process can be computationally intensive. The paper mentions that the WPT coefficients are used for filtering framewise, implying real-time processing is a consideration.
  • Thresholding for Late Reflection: The method relies on thresholding wavelet coefficients to isolate late reflections. The effectiveness of this thresholding might be sensitive to the signal characteristics.
  • Future Work: The paper suggests further exploration of different entropy-based criteria and potentially more advanced acoustic models for WPT selection. The tight integration of dereverberation with ASR is a promising avenue for future research.

Summary updated

BibTeX
@inproceedings{gomez2012dereverberation,
  author = {Randy Gomez and Tatsuya Kawahara},
  title = {{Dereverberation based on wavelet packet filtering for robust automatic speech recognition}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure