TL;DR
This paper introduces a novel iterative algorithm for cleaning up speech recorded in reverberant environments using multiple microphones. By combining a multi-channel Wiener filter with a codebook-based constraint and a long-term prediction method, the algorithm effectively suppresses late reverberations, leading to improved speech quality and intelligibility.
What Problem Does This Paper Address?
Hands-free speech systems, like those in conference rooms or mobile devices, often suffer from reverberation. This occurs when sound waves bounce off surfaces in a room, creating echoes that smear the original speech signal. This smearing significantly degrades speech intelligibility, making it harder for both humans and automatic speech recognition (ASR) systems to understand. While single-microphone solutions exist, multi-microphone systems theoretically offer better dereverberation. However, effectively leveraging multiple microphones to remove the complex late reverberations (which are the primary cause of intelligibility loss) remains a challenging problem.
Methods
The core of the proposed solution is a Multi-channel Iterative Dereverberation (MID) algorithm. It's built upon two key components:
- Codebook Constrained Iterative Multi-channel Wiener Filter (CCIMWF): This extends the traditional iterative Wiener filter (IWF) to handle multiple microphone inputs (MIMO). The "codebook constrained" aspect is crucial: it uses a pre-trained codebook of clean speech characteristics to impose intra-frame constraints. This helps the filter converge better and reduces artifacts. The CCIMWF operates on short frames of speech.
- Long-Term Multi-step Linear Prediction (LTMLP): This technique is used to estimate the late reverberation component of the speech signal. It analyzes the speech over longer durations (utterances) to predict these delayed reflections.
The MID algorithm employs a doubly iterative approach. An outer loop alternates between estimating late reverberations using LTMLP and then refining the speech using the CCIMWF. The CCIMWF itself also involves inner iterations. This iterative process allows the algorithm to progressively suppress reverberations and improve the signal-to-reverberation ratio (SRR).
Datasets
The experiments were conducted using a simulated 4-channel reverberant speech setup. The room impulse response was generated using the image method for a specific room size and source/microphone positions. The clean speech codebook was generated using approximately 38 minutes of clean speech from the TIMIT database (dialect region 2). The feature vectors for the codebook were calculated over 20ms frames.
Key Insights and Ideas
- Leveraging Multi-channel Information: The paper demonstrates that multi-channel processing offers a significant advantage for dereverberation, as theoretically perfect removal is possible under certain assumptions.
- Codebook Constraints are Key: The inclusion of a clean speech codebook in the Wiener filter (CCIMWF) significantly improves convergence and performance compared to a standard multi-channel Wiener filter (IMWF) without such constraints. This highlights the importance of incorporating prior knowledge about clean speech.
- Doubly Iterative Refinement: The outer loop of LTMLP for reverberation estimation and CCIMWF for enhancement is effective. It allows for progressive improvement by refining both the reverberation estimate and the enhanced speech signal iteratively.
- Independent Channel Processing is Better: The CCIMWF, which allows independent quantization for each channel, performed better than the joint-CCIMWF (j-CCIMWF), where all channels were constrained to quantize to the same codebook vector. This suggests that individual channel characteristics, even in a multi-channel setup, are important.
Key Results
- The algorithm showed significant improvements in Average Segmental Signal-to-Reverberation Ratio (SSRR) across all four channels.
- The Average Segmental Log Spectral Distortion (SLSD), a measure related to intelligibility, decreased for the proposed CCIMWF and j-CCIMWF methods, indicating improved speech quality.
- More than 70% of speech frames converged within 3 iterations of the CCIMWF.
- The proposed CCIMWF and j-CCIMWF methods consistently outperformed the baseline IMWF (without codebook constraints), demonstrating the effectiveness of the codebook approach.
- The improvement in SSRR was marginal after the 3rd MID iteration, suggesting an optimal number of iterations for practical use.
Limitations and Future Work
- No Explicit Convergence Criterion: The authors note that they do not have an explicit convergence criterion for the MID and CCIMWF iterations, relying instead on experimental determination of the optimal number of iterations.
- Parameter Sensitivity: While the paper mentions that the parameter $\mu$ (weighting speech distortion vs. noise energy in the Wiener filter) is not critical, the optimal number of iterations for best performance was determined experimentally.
- Future Work: The paper suggests that the CCIMWF formulation, while used here for dereverberation, could be applied to other multi-channel speech enhancement applications. The authors also imply that further exploration of joint processing or alternative constraint mechanisms might be beneficial.
Summary updated