A signal-separation-based array postfilter for distant speech recognition

Rita Singh, Kenichi Kumatani, John McDonough, Chen Liu2012 · DOI

TL;DR

This paper introduces a novel postfiltering technique for distant speech recognition that treats the problem as a signal separation task rather than traditional noise filtering. By leveraging a compositional model, their method effectively disentangles speech from noise without making assumptions about the noise characteristics, leading to significantly improved word error rates compared to existing methods.

What Problem Does This Paper Address?

In scenarios where microphones are distant from the speaker (e.g., in a car), speech recognition systems often struggle with background noise. Microphone arrays help by spatially filtering the audio to focus on the speaker's direction (beamforming). However, the output of the beamformer still contains residual noise. Traditional methods, known as postfilters, aim to remove this remaining noise, often using Wiener filters. These filters rely on assumptions about the noise (e.g., being diffuse or uncorrelated across microphones) to estimate clean speech. When these assumptions don't hold, postfiltering performance degrades, especially with small microphone arrays where robust estimation is difficult. This paper tackles the challenge of robustly removing residual noise from beamformed speech signals, particularly in non-ideal noise conditions and with limited microphone arrays.

Methods

The core of this paper's contribution is a compositional postfiltering mechanism that reframes the problem as semi-supervised source separation. Instead of trying to filter out noise based on its properties, it aims to explicitly separate the speech and noise components.

Here's a breakdown:

  • Compositional Model: They adapt a probabilistic latent component analysis (PLCA) model, typically used for monaural source separation. This model treats spectral vectors of speech and noise as histograms drawn from mixture-multinomial distributions. Each distribution represents "compositional units" (e.g., spectral patterns) that make up the speech or noise.
  • Semi-Supervised Separation: The key is that they have access to the mixed signal (the beamformer output, y[t]) and can obtain an estimate of the noise component (n[t]) separately.
  • Noise Estimation: To get n[t], they employ a null-steering beamformer. This specialized beamformer is designed to "scan" the environment, identify the direction of the strongest interfering signal (while nulling the target speaker), and thus provide a noise-only signal.
  • Learning Components:
    • Noise Components (P_N(f|z)): These are learned from the estimated noise signal (n[t]) using standard EM algorithms.
  • Speech Components (P_S(f|z)) and Mixture Weights (P_t(S), P_t(z), P_t(N), P_N(z)): These are learned iteratively from the beamformer output (y[t]) using a modified EM algorithm. This is the "semi-supervised" aspect, as they don't have clean speech examples to train on directly.
  • Postfilter Design: Once the speech and noise components are learned, a time-varying postfilter H_t(f) is computed. This filter essentially acts as a Wiener filter, but its parameters are derived from the learned speech and noise distributions, effectively separating the speech from the residual noise in the beamformer output.

What's New: The novelty lies in applying a compositional source separation framework to the postfiltering problem, specifically by using a null-steering beamformer to obtain a noise estimate and then iteratively learning speech and noise components. This avoids the explicit assumptions about noise characteristics required by traditional Wiener postfilters.

Datasets

The experiments were conducted on a corpus of in-car two-channel recordings. These recordings were made under eight different operating conditions, simulating various driving states (idle, highway speeds) and environmental factors (engine running, fan on, turn signal, passenger window open). The data was collected using a two-microphone array and a close-talking headset microphone for each speaker. The speech recognition system used was the CMU Sphinx-3 ASR system, with acoustic models trained on the WSJ1 corpus and a language model trained on WSJ1 transcriptions.

Key Insights and Ideas

  • Separation vs. Filtering: The fundamental insight is that treating postfiltering as a signal separation problem, rather than just noise filtering, is more effective. This allows the system to explicitly model and remove noise without inadvertently attenuating speech.
  • Assumption-Free Noise Modeling: The proposed method's strength lies in its ability to handle various noise conditions without requiring prior assumptions about the noise's statistical properties (e.g., diffuse, uncorrelated). This makes it more robust in real-world scenarios.
  • Adaptability and Unsupervised Learning: The compositional model is adaptive on an utterance-by-utterance basis. Crucially, it's largely unsupervised for the speech component, meaning it doesn't require pre-recorded clean speech samples for training, making it practical.
  • Conservatism in Speech Preservation: The authors highlight that their method is "conservative," actively preserving speech while suppressing noise. This contrasts with traditional filters that might remove speech components along with noise, leading to degradation.
  • Array Size Independence: Unlike some traditional postfilters that rely on averaging across multiple microphones for robust parameter estimation, this method primarily uses the beamformer output, reducing its dependence on array size.

Key Results

The paper reports significant improvements in word error rates (WER) using their proposed compositional postfilter.

  • Outperforms Baselines: Table 1 shows that their "Comp. PF" consistently achieves lower WERs across all tested states compared to baseline methods like "Close-talking," "Farfield Channel 2," "MNES WOPF" (beamforming without postfiltering), and other post-filtering methods like "Leukim," "McCowan," and "Zelinski."
  • Significant Reductions: For instance, in the "35 mph, Wind" condition, the proposed method achieves a WER of 58.4%, while the next best postfilter (McCowan) is at 64.8%, and the beamformer alone is 65.9%. In the challenging "65 mph, Wind" condition, their method achieves 45.6% WER, significantly better than the next best at 50.6%.
  • Qualitative Improvement: The method is described as actively preserving speech while suppressing noise, leading to cleaner audio for the ASR system.

Limitations and Future Work

  • Iterative Refinement: The authors note that while their current implementation is a batch-processing module, an iterative application of the compositional postfilter might yield further improvements, as the postfiltered signal still contains some residual noise.
  • Basic Compositional Models: The current compositional models are described as "quite basic." Future work could involve developing more sophisticated sparsity constraints and overcomplete basis representations.
  • Better Update Rules: Improving the update rules for estimating the mixture weights is also suggested as a direction for future research.
  • Null-Steering Beamformer: While effective, the null-steering beamformer's ability to perfectly isolate noise might be a point for further investigation and refinement.

Summary updated

BibTeX
@misc{singh2012signal,
  author = {Rita Singh and Kenichi Kumatani and John McDonough and Chen Liu},
  title = {{A signal-separation-based array postfilter for distant speech recognition}},
  year = {2012},
  doi = {10.21437/interspeech.2012-524},
}
Figure