TL;DR
This paper introduces a novel "Two-Step Band-Split Neural Network" (TBNN) for acoustic echo cancellation (AEC) that significantly improves performance by splitting the audio signal into wideband and highband frequencies. This approach, which achieved second place in the ICASSP 2023 AEC Challenge, tackles the complexity of full-band echo suppression by using specialized neural networks for each frequency range.
What Problem Does This Paper Address?
Acoustic echo cancellation (AEC) is crucial for clear communication, especially in full-duplex systems like video conferencing or phone calls. The challenge lies in removing the echo of the far-end speaker's voice that is picked up by the near-end microphone, without distorting the near-end speaker's voice or removing other important audio components. Traditional AEC methods often struggle with full-band audio (up to 48kHz), where different frequency ranges have distinct characteristics: wideband frequencies (e.g., 16kHz) contain more structured speech information, while highband frequencies (16-48kHz) are more prone to noise and less structured. Directly modeling this full-band signal with a single neural network can be computationally expensive and less effective. This paper aims to address this by proposing a more efficient and effective method for full-band echo suppression.
Methods
The proposed approach, the Two-Step Band-Split Neural Network (TBNN), first employs a linear filter to remove the bulk of the echo. Then, it splits the residual echo signal into two bands: a wideband signal (up to 16kHz) and a highband signal (16-48kHz).
- Wideband Post-Filter (WBPF): This component uses an updated Gated Convolutional Recurrent Network (GCRN) with a U2-Encoder. The U2-Encoder is a novel architecture designed to capture contextual information across different scales efficiently, improving latent feature extraction.
- Highband Post-Filter (HBPF): This component is designed with lower complexity. It utilizes prior information from the WBPF's output and processes the highband signal separately. The HBPF uses 2D-convolutional modules to extract features and a GRU layer to predict a complex-valued mask, which is then applied to the highband input to obtain the enhanced highband output.
- Time Delay Estimation (TDE): A TDE module based on sub-band cross-correlation is used to align the reference signal.
- Loss Function: The loss function is a combination of echo power weighted magnitude loss and a power-law compressed phase aware (PLCPA) loss, calculated separately for wideband and highband signals. Auxiliary VAD (Voice Activity Detection) and masked MSE losses are included to prevent over-suppression of near-end speech.
Datasets
- Training Data: The dataset is a combination of real recordings from the AEC-Challenge far-end single-talk dataset and synthetic data. Synthetic data is generated by convolving room impulse responses (RIRs) with near-end speech from the 4th deep noise suppression (DNS) challenge and noises from Freesound and AudioSet. The training set comprises 2000 hours of real-recorded echo and 1000 hours of simulated echo.
- Validation and Test Sets: These are generated similarly to the training data, with 50 hours for validation and 1500 clips for testing.
Key Insights and Ideas
- Band-splitting is beneficial: Separating the signal into wideband and highband allows for specialized processing, leveraging the different characteristics of each frequency range and reducing overall modeling complexity.
- U2-Encoder for efficiency: The U2-Encoder architecture proves effective in capturing multi-scale contextual information without a significant increase in computational cost, outperforming simply increasing channel numbers in the network.
- Masking outperforms mapping in HBPF: For the highband post-filter, using a masking approach to estimate the output yields better near-end speech quality compared to a mapping approach.
- Prior information from WBPF helps HBPF: Providing the wideband output as prior information to the highband post-filter improves its performance.
Key Results
The TBNN approach achieved a strong performance in the ICASSP 2023 AEC Challenge, ranking 2nd place (tied) in the non-personalized track.
- Overall Mean Opinion Score (MOS): 4.344
- Word Accuracy (WAcc) ratio: 0.795
- Real-Time Factor (RTF): 0.35 (on Intel(R) Xeon(R) E5-2678 v3 @2.50GHz), with 0.28 for the TBNN post-filter and 0.07 for the TDE and adaptive filter.
Limitations and Future Work
- Limitations: The paper does not explicitly detail limitations, but the reliance on a two-step process and specialized networks for each band might introduce some latency or complexity in deployment compared to a single end-to-end model. The performance on highly reverberant or noisy conditions might also be a point for further investigation.
- Future Work: The authors suggest that enlarging the model size (as demonstrated by TBNN-L-masking) leads to better performance, indicating potential for further improvements with larger models. Exploring more advanced encoder-decoder architectures or different band-splitting strategies could also be future research directions.
Summary updated