Introduction
Reverberation—the persistence of sound in an enclosed space—poses a significant challenge for users of hearing aids (HA) and cochlear implants (CI). While natural for normal-hearing listeners, excessive reverberation smears speech signals, severely degrading intelligibility and increasing cognitive load. A common signal processing approach to mitigate this is Weighted Prediction Error (WPE), which utilizes a linear prediction filter to estimate and subtract late reflections.
However, traditional WPE implementations often rely on isolated components that are not optimized for the final audio output. This paper presents an end-to-end (E2E) optimization framework for online speech dereverberation. By training a neural network to optimize the final processed signal rather than an intermediate spectral estimate, the researchers achieved superior speech quality tailored to the specific needs of HA and CI users.
The Challenge: The "Mismatch" Problem
Standard WPE requires an accurate estimate of the clean speech's Power Spectral Density (PSD). Modern approaches often use Deep Neural Networks (DNNs) to provide this estimate. However, these DNNs are typically trained in isolation to minimize spectral error. This creates a "mismatch": the network is optimized to predict a spectrum, but the ultimate goal of the hearing device is to produce high-quality, intelligible audio.
Furthermore, different medical devices have different requirements. HA users often benefit from "early reflections" which assist with spatial awareness and localization. In contrast, CI users, who process sound through electrical stimulation of the auditory nerve, often find these reflections harmful to clarity. Traditional dereverberation methods struggle to adapt to these distinct user requirements without significant manual tuning.
Proposed Method: End-to-End Online DNN-WPE
The authors propose a system that integrates a Recurrent Neural Network (LSTM) directly into the WPE processing chain. The key components include:
1. Recursive Least Squares (RLS)-WPE
To function in real-time, the system uses an online version of WPE. The RLS algorithm allows the filter to adapt frame-by-frame to changing room acoustics, which is essential for mobile hearing device users.
2. End-to-End Training
Instead of training the LSTM to match a "clean" PSD, the authors backpropagate the error through the entire WPE processing chain. The loss function (Kullback-Leibler divergence) is calculated at the final output of the system. This allows the DNN to learn a "pseudo-PSD" that acts as a regularizer, correcting for the mathematical assumptions (such as Gaussianity) that the standard WPE algorithm makes.
3. Customizable Targets
The framework allows for simple customization of the training target signal ($\nu$):
- For HA users: The target includes the direct path plus early reflections ($\nu = d + e$).
- For CI users: The target is strictly the direct path ($\nu = d$).
4. Addressing the Initialization Gap
RLS filters require a "warm-up" period to converge. The researchers observed that performance is significantly lower during the first 4 seconds of audio. To prevent this instability from degrading the learning process, they developed a training procedure that excludes this initialization period from gradient updates, ensuring the network optimizes for steady-state performance.
Experimental Setup and Datasets
The study utilized a noise-free version of the WHAMR! dataset, combining WSJ0 anechoic speech with realistic 2-channel Room Impulse Responses (RIRs) generated by the RAZR engine. Reverberation times ($T_{60}$) ranged from 0.4 to 1.0 seconds. The training set comprised approximately 55 hours of speech sampled at 16 kHz.
Key Results and Insights
The proposed E2E-WPE-p (pretrained and fine-tuned) model consistently outperformed both "Vanilla" RLS-WPE and standard DNN-supported WPE across multiple metrics:
- Speech Quality (PESQ): In the HA scenario, the E2E approach achieved a score of 2.76, outperforming the standard DNN-WPE (2.72) and even matching "Oracle" versions in high-reverberation environments.
- Intelligibility (ESTOI): The system showed consistent improvements in intelligibility across varying $T_{60}$ times.
- Real-time Feasibility: The system maintains a real-time factor below 0.1 on a standard GPU, indicating it is computationally efficient enough for low-latency hearing device applications.
- Visual Analysis: Spectrograms confirmed that the E2E approach effectively suppresses the "smearing" of speech energy into silent intervals more cleanly than previous methods.
Conclusion and Future Work
This research demonstrates that training neural networks to optimize the final output of a signal processing chain—rather than intermediate steps—yields significant gains in audio quality. By accounting for the specific needs of HA and CI users and addressing the practicalities of online filter convergence, the authors have created a robust framework for real-world dereverberation.
Future work will focus on extending this framework to noisy environments, where simultaneous denoising and dereverberation are required. Additionally, the authors suggest that adding a noise-reduction post-processing module could further refine the output, as residual late reverberation often mimics the perceptual characteristics of background noise.
Summary updated