Multichannel Online Dereverberation Based on Spectral Magnitude Inverse Filtering

Xiaofei Li, Laurent Girin, Sharon Gannot, Radu Horaud

In the world of audio processing, reverberation is the "smearing" of sound caused by reflections off walls and objects. While a bit of reverb makes a concert hall sound grand, too much of it makes speech unintelligible for humans and disastrous for automatic speech recognition (ASR) systems.

This paper presents a robust, real-time solution for multichannel dereverberation, specifically designed to handle the "nightmare scenario" of a speaker moving around a room.

TL;DR

The authors propose Spectral Magnitude Inverse Filtering (SMIF), an online multichannel dereverberation method that operates in the STFT magnitude domain. By focusing on signal magnitudes rather than complex values, the system achieves remarkable robustness against filter estimation errors, significantly outperforming state-of-the-art methods in dynamic environments with moving speakers.

What Problem Does This Paper Address?

Dereverberation is traditionally handled by identifying the room's impulse response and "inverting" it. However, doing this online (as the audio is captured) and for moving speakers is incredibly difficult.

Standard complex-domain methods like the Multiple Input/Output Inverse Theorem (MINT) are mathematically precise but extremely fragile. Even a tiny error in estimating the room filters—common when a speaker is moving—can lead to massive distortions or "musical noise." The authors aim to bridge the gap between theoretical performance and real-world robustness.

Methods

The core innovation is moving the inverse filtering process from the complex domain to the nonnegative magnitude domain.

  1. Adaptive CTF Identification: The system first identifies the subband Convolutive Transfer Function (CTF) for each microphone channel. It uses a Recursive Least Squares (RLS)-like algorithm (Algorithm 1) for fast convergence, which is essential for tracking a moving sound source. Algorithm 1: Recursive estimation of the covariance matrix inverse for CTF identification.
Algorithm 1: Recursive estimation of the covariance matrix inverse for CTF identification.
  1. Magnitude Convolution Model: Instead of the exact complex-valued convolution, the authors use a coarse approximation: the magnitude of the microphone signal is roughly the convolution of the source magnitude and the CTF magnitude. While less precise, this model is far more robust to small phase and timing perturbations.
  2. Online Magnitude MINT: They adapt the MINT equalization principle to this magnitude model. The inverse filters are updated frame-by-frame using a normalized gradient descent (LMS-like) approach (Algorithm 2). Algorithm 2: The adaptive magnitude inverse filtering process.
Algorithm 2: The adaptive magnitude inverse filtering process.
  1. Phase Reconstruction: Since the filtering happens in the magnitude domain, the original phase from one of the microphones is used to reconstruct the final time-domain signal.

Datasets

The authors evaluated their method using two primary benchmarks:

  • REVERB Challenge Dataset: Includes both simulated data (SimData) with a $T_{60}$ of 0.7s and real-world recordings (RealData) in noisy, reverberant rooms.
  • Dynamic Dataset: A specialized dataset featuring human speakers who are either static (facing or not facing the array) or moving between different points in the room.

Key Insights and Ideas

  • Robustness over Precision: The "aha moment" is that a coarse approximation in the magnitude domain is actually better for real-world applications than a "perfect" complex-domain model. By ignoring phase-sensitive interference patterns during the inversion process, the algorithm avoids the catastrophic artifacts that plague traditional MINT.
  • RLS for Tracking: Using RLS instead of standard LMS for CTF identification allows the system to "re-learn" the room acoustics almost instantly as a speaker moves, providing a significant edge in dynamic scenarios.
  • Pairwise Processing: For many-microphone setups, the authors found that averaging the results of pairwise magnitude inversions (processing pairs of microphones independently) further reduces distortions.

Key Results

The proposed SMIF method was compared against the popular Weighted Prediction Error (WPE) algorithm.

  • Speech Quality: On the REVERB dataset, SMIF showed competitive results in SRMR, PESQ, and STOI metrics. Table I: Performance metrics on the REVERB challenge dataset.

  • ASR Performance: On the RealData subset of the REVERB challenge, SMIF achieved a relative Word Error Rate (WER) improvement of approximately 35% compared to the unprocessed signal.

  • Dynamic Robustness: In the Dynamic dataset, SMIF significantly outperformed WPE for moving speakers, maintaining high intelligibility (STOI) where other methods faltered.
  • Efficiency: The system is computationally efficient, with a real-time factor (RF) well below 1.0 for 2-channel configurations, making it suitable for edge devices like smart speakers.
Table I: Performance metrics on the REVERB challenge dataset.

Limitations and Future Work

  • Dereverberation Only: The current method focuses solely on removing echoes and does not explicitly perform noise reduction. In very noisy environments, the residual noise can still impact ASR.
  • Phase Handling: Using the "noisy" microphone phase is a simple heuristic. The authors suggest that integrating more sophisticated phase estimation could further improve quality.
  • Future Directions: The team plans to investigate noise-robust versions of the algorithm and explore tighter integration with deep-learning-based ASR back-ends.

Summary updated

Abstract

This paper addresses the problem of multichannel online dereverberation. The proposed method is carried out in the short-time Fourier transform (STFT) domain, and for each frequency band independently. In the STFT domain, the time-domain room impulse response is approximately represented by the convolutive transfer function (CTF). The multichannel CTFs are adaptively identified based on the cross-relation method, and using the recursive least square criterion. Instead of the complex-valued CTF convolution model, we use a nonnegative convolution model between the STFT magnitude of the source signal and the CTF magnitude, which is just a coarse approximation of the former model, but is shown to be more robust against the CTF perturbations. Based on this nonnegative model, we propose an online STFT magnitude inverse filtering method. The inverse filters of the CTF magnitude are formulated based on the multiple-input/output inverse theorem, and adaptively estimated based on the gradient descent criterion. Finally, the inverse filtering is applied to the STFT magnitude of the microphone signals, obtaining an estimate of the STFT magnitude of the source signal. Experiments regarding both speech enhancement and automatic speech recognition are conducted, which demonstrate that the proposed method can effectively suppress reverberation, even for the difficult case of a moving speaker.
BibTeX
@article{li2019multichannel,
  author = {Xiaofei Li and Laurent Girin and Sharon Gannot and Radu Horaud},
  title = {{Multichannel Online Dereverberation Based on Spectral Magnitude Inverse Filtering}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2019},
  url = {https://openreview.net/forum?id=5d1ih4griVR},
}

Loading related content...

Figure