Abstract
This paper addresses the problem of multichannel online dereverberation. The proposed method is carried out in the short-time Fourier transform (STFT) domain, and for each frequency band independently. In the STFT domain, the time-domain room impulse response is approximately represented by the convolutive transfer function (CTF). The multichannel CTFs are adaptively identified based on the cross-relation method, and using the recursive least square criterion. Instead of the complex-valued CTF convolution model, we use a nonnegative convolution model between the STFT magnitude of the source signal and the CTF magnitude, which is just a coarse approximation of the former model, but is shown to be more robust against the CTF perturbations. Based on this nonnegative model, we propose an online STFT magnitude inverse filtering method. The inverse filters of the CTF magnitude are formulated based on the multiple-input/output inverse theorem (MINT), and adaptively estimated based on the gradient descent criterion. Finally, the inverse filtering is applied to the STFT magnitude of the microphone signals, obtaining an estimate of the STFT magnitude of the source signal. Experiments regarding both speech enhancement and automatic speech recognition are conducted, which demonstrate that the proposed method can effectively suppress reverberation, even for the difficult case of a moving speaker.
Introduction
Reverberation—the persistence of sound after the source has stopped due to reflections—poses a significant challenge for audio processing. While it can add aesthetic quality to music, excessive reverberation severely degrades speech intelligibility and the performance of automatic speech recognition (ASR) systems. Traditional dereverberation methods often struggle in real-time, "online" scenarios, particularly when the sound source is moving, as the room's acoustic response changes dynamically.
This paper introduces Spectral Magnitude Inverse Filtering (SMIF), a robust multichannel approach designed to handle these dynamic environments by operating in the STFT magnitude domain rather than the complex domain.
Methodology
Adaptive CTF Identification
The system first identifies the subband Convolutive Transfer Function (CTF) for each microphone channel. To ensure the system can track a moving speaker, the authors employ a Recursive Least Squares (RLS) criterion. This allows for fast convergence and continuous adaptation to changing acoustic paths.
Magnitude Inverse Filtering
The core innovation of SMIF is the use of a nonnegative convolution model. Traditional methods like the Multiple Input/Output Inverse Theorem (MINT) are highly sensitive to phase and timing errors. By focusing on the STFT magnitude, the proposed method sacrifices some mathematical precision for significant gains in robustness against filter estimation errors.
The inverse filters are updated frame-by-frame (as shown in Algorithm 2) and applied to the microphone signal magnitudes. The final time-domain signal is reconstructed using the original phase from one of the microphones, a heuristic that proves effective in maintaining signal integrity without the artifacts common in complex-domain inversion.
Experimental Results
The SMIF method was evaluated using the REVERB Challenge dataset and a specialized Dynamic Dataset featuring moving speakers.
Speech Quality and Intelligibility
As shown in Table I, SMIF (and its variants SMIF-MC/PW) provides competitive results across several metrics, including Speech-to-Reverberation Modulation Energy Ratio (SRMR), Perceptual Evaluation of Speech Quality (PESQ), and Short-Time Objective Intelligibility (STOI). Notably, SMIF maintains high performance in real-world recordings (RealData) where traditional models often falter.
ASR and Dynamic Robustness
In ASR tests on the REVERB RealData subset, SMIF achieved a relative Word Error Rate (WER) improvement of approximately 35%. Furthermore, in scenarios with moving speakers, SMIF significantly outperformed the popular Weighted Prediction Error (WPE) algorithm, demonstrating its superior tracking and robustness capabilities. The system is also computationally efficient, achieving a real-time factor well below 1.0 for 2-channel setups.
Conclusion
By shifting the dereverberation process to the magnitude domain and utilizing adaptive RLS-based identification, the proposed SMIF method offers a practical and robust solution for real-time audio cleaning. It effectively bridges the gap between theoretical inverse filtering and the unpredictable nature of real-world acoustic environments, particularly for mobile sound sources.
Summary updated