A cepstral domain maximum likelihod beamformer for speech recognition

Dominik Raub, John McDonough, Matthias WöfelInterspeech 2004

TL;DR

This paper introduces a novel beamforming technique that optimizes for speech recognition accuracy (Word Error Rate - WER) rather than just signal quality. By adapting beamformer weights directly to the acoustic models used by the speech recognizer, they achieve significant improvements in recognition performance, especially in noisy and reverberant environments.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems perform well in controlled environments but struggle with distant microphones due to noise and reverberation. Traditional beamforming techniques, designed to maximize Signal-to-Noise Ratio (SNR) or maintain a distortionless response, don't directly translate to lower WER. This is because the optimal beamformer for speech recognition isn't necessarily the one that sounds "cleanest" or has the highest SNR; it's the one that produces features most compatible with the ASR's acoustic models. This paper tackles this mismatch by proposing a beamformer whose optimization criterion is directly aligned with the ASR's goal: minimizing word errors.

Methods

The core innovation lies in adapting Seltzer's Maximum Likelihood (ML) beamforming approach. Instead of using log-spectral domain auxiliary models, this paper proposes using Cepstral Domain Maximum Likelihood (CDML) beamforming. This means the beamformer weights are optimized to maximize the likelihood of the cepstral features (which are what the ASR system actually uses) under the ASR's own acoustic models.

Key techniques include:

  • Cepstral Domain Auxiliary Models: The paper replaces Seltzer's log-spectral auxiliary models with cepstral domain models. This is crucial because it directly aligns the beamformer's output with the ASR's input features.
  • Gradient-Based Optimization: The beamformer weights are updated using gradient descent methods, driven by the derivative of the error function (negative log-likelihood) with respect to the weights.
  • Generalized Sidelobe Canceller (GSC): The GSC framework is used to handle the optimization. It allows for unconstrained optimization of active weights while maintaining a quiescent weight vector that enforces desired signal characteristics.
  • Modified GSC-CDML Beamformer: To address reverberation, the authors propose a modified GSC that incorporates tap-delay lines. This allows for longer impulse responses to better cancel reverberant components without distorting the desired signal.

Datasets

  • English Spontaneous Scheduling Task (ESST) corpus: Used for training both the ASR recognizer and the auxiliary models for beamforming. This corpus consists of approximately 35 hours of dialog from 242 speakers, recorded in a meeting room scenario.
  • ESST test set: Used for evaluation, comprising 1,825 utterances from 16 speakers, totaling 210 minutes.
  • Interference: Speech from another talker and music from a chamber orchestra were added to the "interference-free" signal to simulate noisy conditions.

Key Insights and Ideas

  • The "Mismatch" is Key: The fundamental insight is that optimizing for SNR or distortionless response is suboptimal for ASR. The beamformer must be optimized with respect to the ASR's internal acoustic models to achieve lower word error rates.
  • Cepstral Domain Alignment: Directly optimizing in the cepstral domain, where the ASR operates, is more effective than using log-spectral domain approximations. This is because cepstral coefficients are inherently correlated across frequency bands, making direct optimization more relevant.
  • Reverberation is a Major Hurdle: The paper acknowledges reverberation as a primary challenge in distant microphone ASR. Their proposed tap-delay line GSC aims to mitigate this by allowing for longer impulse responses.
  • GSC's Flexibility: The GSC framework proves useful for incorporating external information (like visual speaker location) or for managing the optimization process in a more flexible way.

Key Results

  • Significant WER Reduction: CDML beamforming achieved improvements of up to 2.29% absolute WER in the presence of interference compared to baseline methods.
  • GSC-CDML Performance: The GSC-CDML beamformer (which acts more as a filter than a postfilter) performed only slightly worse (0.94% absolute) than the CDML beamformer, suggesting the postfiltering effect of CDML contributes to the gains.
  • Correlation between Likelihood and WER: The authors found that improvements and degradations in cepstral domain likelihood generally correlated well with improvements and degradations in WER. However, they struggled to establish a reliable connection between log-spectral domain likelihoods and WER in their experiments.

Limitations and Future Work

  • Convergence Issues: The paper notes that due to the correlated nature of cepstral coefficients, independent optimization of weights (as done in log-spectral domain) is not feasible, potentially leading to convergence issues.
  • Reverberation Mitigation: While the tap-delay line approach is promising, the paper suggests further experiments are needed to fully exploit its potential for reverberation.
  • Filterbank Overlap: The use of an FFT filterbank leads to significant overlap between frequency bins, potentially violating the independence assumption and impacting performance. Employing filterbanks with better separation properties is suggested.
  • Tracking Capability: The current semi-supervised and unsupervised ML beamformers are calibrated once or adapt slowly. The authors highlight the need for beamformers with true tracking capabilities to handle rapidly changing environmental conditions, suggesting algorithms like LMS, RMS, or Kalman filters. Preliminary experiments with LMS were too slow, while RLS and Kalman filters showed promise in related work.

Summary updated

BibTeX
@inproceedings{raub2004cepstral,
  author = {Dominik Raub and John McDonough and Matthias Wöfel},
  title = {{A cepstral domain maximum likelihod beamformer for speech recognition}},
  booktitle = {Interspeech},
  year = {2004},
}
Figure