Template-based spectral estimation using microphone array for speech recognition

Satoshi Tamura, Eriko Hishikawa, Wataru Taguchi, Satoru HayamizuInterspeech 2010

TL;DR

This paper introduces a novel microphone array signal processing technique called Template-based Spectral Estimation (TSE) to improve speech recognition in noisy environments. TSE uses a "noise template" to better estimate the target speech signal, outperforming existing methods, especially when combined with Spectral Subtraction (SS).

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems struggle in noisy real-world environments, significantly degrading their accuracy. While various noise reduction techniques exist, they often fall short, especially with specific types of noise where traditional methods like Spectrum Subtraction (SS) or Complex Spectrum Circle Centroid (CSCC) falter. The challenge is to effectively separate the desired speech signal from interfering noise, particularly when the noise characteristics are complex or when traditional assumptions about signal geometry are violated.

Methods

The paper proposes a Template-based Spectral Estimation (TSE) method. The core idea is to create a "noise template" for each frequency bin using non-speech audio signals observed at the microphones. This template represents the expected spatial and spectral characteristics of the noise. Then, for noisy speech signals, TSE uses this template and a gradient descent method to estimate the clean speech spectrum.

This differs from the Complex Spectrum Circle Centroid (CSCC) method, which assumes that noisy signals at different microphones lie on a circle in the complex plane and estimates the target as the circle's centroid. TSE is more robust because it doesn't strictly require this circular assumption and can handle situations where signals lie on a line (e.g., when phase differences are minimal).

The authors also propose a combined approach: SS-TSE. Here, Spectral Subtraction (SS) is applied first to each microphone signal individually, followed by the TSE method. This combination aims to leverage the noise reduction capabilities of SS while using TSE to compensate for spectral distortions introduced by SS and to handle challenging noise scenarios.

Datasets

  • AURORA-2J test set: Used for evaluating both noise reduction and speech recognition performance. This dataset is a standard benchmark for noisy speech recognition.
  • AURORA-2J training set: Used for training acoustic models.
  • Clean speech data: Used for model training in the speech recognition experiments.

The paper does not introduce a new dataset but uses a specific type of "pure" random noise for experiments, characterized by large magnitude in frequency bins with near-zero phase differences between microphones.

Key Insights and Ideas

  • The limitation of geometric assumptions: The CSCC method's reliance on signals forming a circle in the complex plane makes it vulnerable to noise with minimal phase differences across microphones, where signals tend to align linearly.
  • The power of a "noise template": TSE's ability to learn and adapt to noise characteristics via a template allows it to handle more complex noise scenarios than methods relying solely on geometric properties.
  • Synergy between SS and TSE: Combining SS (for initial noise reduction) with TSE (for spectral distortion compensation and robust estimation) yields superior results for both noise reduction and speech recognition compared to either method alone.
  • Phase matters: The paper highlights that phase differences (or lack thereof) between microphone signals are critical for the performance of different noise reduction techniques.

Key Results

  • Noise Reduction: The SS-TSE method achieved the highest Noise Reduction Rate (NRR) of 16.7dB. The D-spec values (average error of amplitude spectrum) showed a similar trend, indicating SS-TSE's effectiveness.
  • Speech Recognition: The SS-TSE method achieved the best word recognition accuracy at 59.37%. This represents a significant improvement over other methods, including the SS-CSCC combination (51.89%) and individual methods like DS (15.18%), SS (27.26%), CSCC (36.75%), and TSE (37.01%).
  • Error Reduction: The SS-TSE method resulted in a 44% relative recognition error reduction compared to the conventional SS method.

Limitations and Future Work

  • Limitations:
    • The CSCC and TSE methods struggled with the specific "pure" random noise used in the experiment due to minimal phase differences between microphones, causing spectra to align linearly rather than circularly.
  • The SS method, while good at noise reduction, can introduce spectral distortions that negatively impact speech recognition.
  • Future Work:
    • Adapting the proposed method to handle multiple sound sources.
  • Investigating optimal frequency band widths for improved recognition performance.
  • Conducting evaluation experiments in real-world field environments.

Summary updated

BibTeX
@inproceedings{tamura2010template,
  author = {Satoshi Tamura and Eriko Hishikawa and Wataru Taguchi and Satoru Hayamizu},
  title = {{Template-based spectral estimation using microphone array for speech recognition}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure