Constrained multichannel speech dereverberation

Meng Yu, Frank K. SoongInterspeech 2012

TL;DR

This paper introduces a novel multi-channel speech dereverberation technique that leverages cross-channel cancellation and spectrogram decomposition. By using the Itakura divergence and incorporating sparsity constraints, their method effectively reduces reverberation in audio signals, outperforming existing single-channel and simpler multi-channel approaches.

What Problem Does This Paper Address?

Reverberation, the effect of sound waves bouncing off surfaces in an environment, is a major challenge in speech processing. It degrades the intelligibility of speech, making it harder for both humans and machines to understand. While single-channel dereverberation methods exist, they often struggle with ambiguity due to the limited information available from a single microphone. This paper tackles the problem of improving speech dereverberation by utilizing the richer information available from multiple microphones.

Methods

The core of the proposed method lies in a constrained multi-channel spectrogram decomposition. Instead of modeling reverberation as a simple convolution in the time domain, the authors model it in the spectrogram magnitude domain.

Here's a breakdown of their approach:

  • Spectrogram Decomposition: They decompose the reverberant speech spectrogram into a clean speech spectrogram and a deconvolution filter. This is framed as a blind deconvolution problem.
  • Itakura Divergence: Unlike previous methods that used the L2 norm (mean squared error), this paper adopts the Itakura (I-) divergence as the criterion for spectral decomposition. The authors argue that I-divergence is more suitable for spectral estimation, especially when dealing with non-negative spectral data, and it penalizes positive errors more heavily, which can be beneficial for spectral peak fitting.
  • Cross-Channel Cancellation: A key innovation is the incorporation of cross-channel cancellation. This constraint leverages the fact that the clean speech signal should be the same across all microphones, while the reverberation filters (impulse responses) will differ due to spatial separation. By enforcing that the difference between the cross-convolved spectrograms of different channels is minimized, they effectively constrain the deconvolution filters.
  • Sparsity and Non-negativity Constraints: Similar to prior work, they enforce non-negativity on the speech and filter spectrograms, as these are inherently positive. They also incorporate sparsity constraints on the clean speech spectrogram, assuming that speech is sparse in the spectral domain.
  • Split I-divergence with Proportionate Sparsity: To further refine the sparsity control, they introduce a "split I-divergence" method. This involves using an auxiliary variable to enforce sparsity on, while keeping it close to the I-divergence estimate. Crucially, they also propose using proportionate sparsity weights that adapt to the intensity of the spectrogram across different frequency bins. This is motivated by the observation that reverberation is often more intense in lower frequencies.
  • Multiplicative Algorithm: The optimization of these constraints is achieved through a multiplicative update algorithm, which is guaranteed to converge to a stationary point and maintain the non-negativity of the parameters.

Datasets

The authors evaluated their method using:

  • Synthesized reverberant speech signals: These were created by convolving clean speech from the TMIT database with room impulse responses generated using an image-source method. They specifically mention using T60 (reverberation time) values of 0.6s and 0.8s.
  • Real recorded reverberant speech signals: These were captured in a reverberant office room using a loudspeaker and a distant microphone array (X-Box Kinect).

The paper does not introduce a new benchmark dataset but rather uses existing speech data and synthesizes reverberant conditions to test their algorithm.

Key Insights and Ideas

  • Multi-channel information is crucial: The paper strongly suggests that leveraging multiple microphones significantly improves dereverberation performance compared to single-channel methods, primarily by reducing the ambiguity in deconvolution.
  • Itakura Divergence offers advantages: The choice of I-divergence over L2 norm for spectral decomposition is shown to be beneficial for spectral estimation, particularly in handling spectral peaks and non-negative data.
  • Cross-channel cancellation is a powerful constraint: This technique effectively constrains the deconvolution filters by exploiting the spatial differences in reverberation across channels.
  • Sparsity needs careful control: While sparsity is beneficial, simply applying a uniform sparsity constraint can be suboptimal. Adaptive sparsity, like the proportionate sparsity weights proposed, is more effective because reverberation characteristics vary with frequency.
  • The "split" approach balances fidelity and sparsity: The split I-divergence method, combined with an auxiliary variable, allows for a more flexible trade-off between fitting the observed data and enforcing sparsity.

Key Results

The proposed multi-channel dereverberation method, particularly the 4-channel split I-divergence with proportionate sparsity weights, demonstrated significant improvements:

  • Quantitative Metrics:
    • PESQ (Perceptual Evaluation of Speech Quality): Improved from 1.83 to 2.50 (for T60=0.6s) and from 1.70 to 2.62 (for T60=0.8s) on synthesized data. For real recorded data, it improved from 2.01 to 2.32.
  • SRR (Signal to Reverberation Ratio): Improved from -6.94 dB to 4.21 dB (for T60=0.6s) and from -8.02 dB to 3.76 dB (for T60=0.8s) on synthesized data.
  • Qualitative Findings:
    • Spectrogram analysis showed that the proposed method effectively suppressed reverberation tails, especially in low-frequency regions, compared to simpler methods.
  • The method outperformed conventional single-channel methods and simpler multi-channel approaches (like dual-channel L2 and dual-channel I-divergence) in terms of both PESQ and SRR.

Limitations and Future Work

  • Limitations:
    • The paper notes that while the I-divergence method is less sensitive to sparsity weights than L2, it can become less sensitive to sparsity itself, which is addressed by the split method.
  • The performance of the L2 criterion in multi-channel settings can lead to more distortion due to fitting between spectral peaks.
  • Future Work:
    • The authors suggest that the proposed multi-channel framework can be extended to more complex acoustic scenarios and potentially to other signal processing tasks.
  • Further exploration of adaptive sparsity strategies and different divergence measures could lead to even better performance.
  • The paper implicitly suggests that investigating the trade-offs between the number of channels and computational complexity would be a valuable direction.

Summary updated

BibTeX
@inproceedings{yu2012constrained,
  author = {Meng Yu and Frank K. Soong},
  title = {{Constrained multichannel speech dereverberation}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure