Multichannel Online Dereverberation based on Spectral Magnitude Inverse Filtering

Xiaofei Li, Laurent Girin, Sharon Gannot, Radu Horaud

Abstract

This paper addresses the problem of multichannel online dereverberation. The proposed method is carried out in the short-time Fourier transform (STFT) domain, and for each frequency band independently. In the STFT domain, the time-domain room impulse response is approximately represented by the convolutive transfer function (CTF). The multichannel CTFs are adaptively identified based on the cross-relation method, and using the recursive least square criterion. Instead of the complex-valued CTF convolution model, we use a nonnegative convolution model between the STFT magnitude of the source signal and the CTF magnitude, which is just a coarse approximation of the former model, but is shown to be more robust against the CTF perturbations. Based on this nonnegative model, we propose an online STFT magnitude inverse filtering method. The inverse filters of the CTF magnitude are formulated based on the multiple-input/output inverse theorem (MINT), and adaptively estimated based on the gradient descent criterion. Finally, the inverse filtering is applied to the STFT magnitude of the microphone signals, obtaining an estimate of the STFT magnitude of the source signal. Experiments regarding both speech enhancement and automatic speech recognition are conducted, which demonstrate that the proposed method can effectively suppress reverberation, even for the difficult case of a moving speaker.

Introduction

Reverberation—the persistence of sound after the source has stopped due to reflections—poses a significant challenge for audio processing. While it can add aesthetic quality to music, excessive reverberation severely degrades speech intelligibility and the performance of automatic speech recognition (ASR) systems. Traditional dereverberation methods often struggle in real-time, "online" scenarios, particularly when the sound source is moving, as the room's acoustic response changes dynamically.

This paper introduces Spectral Magnitude Inverse Filtering (SMIF), a robust multichannel approach designed to handle these dynamic environments by operating in the STFT magnitude domain rather than the complex domain.

Methodology

Adaptive CTF Identification

The system first identifies the subband Convolutive Transfer Function (CTF) for each microphone channel. To ensure the system can track a moving speaker, the authors employ a Recursive Least Squares (RLS) criterion. This allows for fast convergence and continuous adaptation to changing acoustic paths.

Magnitude Inverse Filtering

The core innovation of SMIF is the use of a nonnegative convolution model. Traditional methods like the Multiple Input/Output Inverse Theorem (MINT) are highly sensitive to phase and timing errors. By focusing on the STFT magnitude, the proposed method sacrifices some mathematical precision for significant gains in robustness against filter estimation errors.

Algorithm 2: The adaptive magnitude inverse filtering process, which updates the inverse filters frame-by-frame using a gradient descent criterion.

The inverse filters are updated frame-by-frame (as shown in Algorithm 2) and applied to the microphone signal magnitudes. The final time-domain signal is reconstructed using the original phase from one of the microphones, a heuristic that proves effective in maintaining signal integrity without the artifacts common in complex-domain inversion.

Experimental Results

The SMIF method was evaluated using the REVERB Challenge dataset and a specialized Dynamic Dataset featuring moving speakers.

Speech Quality and Intelligibility

As shown in Table I, SMIF (and its variants SMIF-MC/PW) provides competitive results across several metrics, including Speech-to-Reverberation Modulation Energy Ratio (SRMR), Perceptual Evaluation of Speech Quality (PESQ), and Short-Time Objective Intelligibility (STOI). Notably, SMIF maintains high performance in real-world recordings (RealData) where traditional models often falter.

Table I: Performance metrics on the REVERB challenge dataset comparing the proposed SMIF methods against baseline and state-of-the-art WPE algorithms.

ASR and Dynamic Robustness

In ASR tests on the REVERB RealData subset, SMIF achieved a relative Word Error Rate (WER) improvement of approximately 35%. Furthermore, in scenarios with moving speakers, SMIF significantly outperformed the popular Weighted Prediction Error (WPE) algorithm, demonstrating its superior tracking and robustness capabilities. The system is also computationally efficient, achieving a real-time factor well below 1.0 for 2-channel setups.

Conclusion

By shifting the dereverberation process to the magnitude domain and utilizing adaptive RLS-based identification, the proposed SMIF method offers a practical and robust solution for real-time audio cleaning. It effectively bridges the gap between theoretical inverse filtering and the unpredictable nature of real-world acoustic environments, particularly for mobile sound sources.

Summary updated

Abstract

This paper addresses the problem of multichannel online dereverberation. The proposed method is carried out in the short-time Fourier transform (STFT) domain, and for each frequency band independently. In the STFT domain, the time-domain room impulse response is approximately represented by the convolutive transfer function (CTF). The multichannel CTFs are adaptively identified based on the cross-relation method, and using the recursive least square criterion. Instead of the complex-valued CTF convolution model, we use a nonnegative convolution model between the STFT magnitude of the source signal and the CTF magnitude, which is just a coarse approximation of the former model, but is shown to be more robust against the CTF perturbations. Based on this nonnegative model, we propose an online STFT magnitude inverse filtering method. The inverse filters of the CTF magnitude are formulated based on the multiple-input/output inverse theorem (MINT), and adaptively estimated based on the gradient descent criterion. Finally, the inverse filtering is applied to the STFT magnitude of the microphone signals, obtaining an estimate of the STFT magnitude of the source signal. Experiments regarding both speech enhancement and automatic speech recognition are conducted, which demonstrate that the proposed method can effectively suppress reverberation, even for the difficult case of a moving speaker.
BibTeX
@article{li2019multichannel,
  author = {Xiaofei Li and Laurent Girin and Sharon Gannot and Radu Horaud},
  title = {{Multichannel Online Dereverberation based on Spectral Magnitude Inverse Filtering}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2019},
  url = {https://openreview.net/forum?id=5d1ih4griVR},
}
Figure