Introduction
In modern teleconferencing and hands-free communication, acoustic echo remains a significant challenge. While traditional solutions rely on linear filters, contemporary hardware often introduces non-linear distortions that degrade their performance. Researchers from Tencent AI Lab have developed NeuralEcho, a unified deep learning framework designed to jointly address acoustic echo cancellation (AEC), noise suppression (NS), and automatic gain control (AGC).
NeuralEcho is a compact, two-stage recurrent neural network (RNN) that leverages cross-channel second-order statistics and self-attention mechanisms. Despite having approximately 70% fewer parameters than state-of-the-art models like F-T-LSTM, it achieves superior performance in speech enhancement and recognition accuracy.
Problem Statement
Standard AEC systems face three primary hurdles in real-world environments:
- Non-linearity: Low-cost speakers and amplifiers warp signals, making the echo a non-linear transformation of the reference signal.
- Background Noise: Systems must distinguish between the target speaker's voice, background noise, and the echo simultaneously.
- Volume Consistency: Post-processing with AGC can inadvertently amplify residual noise if not properly integrated with the enhancement stage.
NeuralEcho addresses these by treating AEC, NS, and AGC as a unified task rather than a sequence of independent modules.
Proposed Method: The NeuralEcho Architecture
The core of NeuralEcho is a two-stage architecture that integrates traditional signal processing concepts into a neural framework.
Two-Stage Processing
- Stage 1 (Initial Estimation): The model processes the microphone signal and far-end reference to calculate Channel, Temporal, and Frequency correlations. A Gated Recurrent Unit (GRU) encoder predicts complex ratio filters (cRF) to generate an initial echo estimate and a preliminary clean signal.
- Stage 2 (Refinement): The model estimates covariance matrices for noise, echo, and target speech. These statistics feed into a Self-Attentive RNN. The Multi-Head Self-Attention (MHSA) mechanism allows the network to dynamically focus on relevant speech features across time frames.
Speaker Awareness and FiLM
To isolate the target user, NeuralEcho can incorporate a speaker embedding. Using Feature-wise Linear Modulation (FiLM), the model biases its filters toward the specific vocal characteristics of the enrolled speaker, enhancing the target voice while suppressing interferers.
Integrated Automatic Gain Control (AGC)
Unlike traditional systems that apply AGC as a post-processing step, NeuralEcho includes a dedicated branch for gain prediction. This branch is trained via multi-task learning alongside the main enhancement task.
By training the AGC jointly, the model learns to be more aggressive in noise suppression for segments that require significant volume boosting, preventing the amplification of artifacts.
Experimental Setup
The model was evaluated using extensive datasets, including AISHELL-2 for clean speech and the AEC-Challenge datasets for synthetic echo and non-linear distortions. Training involved 10,000 simulated Room Impulse Responses (RIRs) with reverberation times up to 0.6s. Testing conditions covered a wide range of Signal-to-Echo Ratios (SER) from -10dB to 10dB and Signal-to-Noise Ratios (SNR) from 0 to 40dB.
Key Results
NeuralEcho demonstrated significant improvements over the F-T-LSTM baseline across all metrics:
| Metric | F-T-LSTM (Baseline) | NeuralEcho (Proposed) | NeuralEcho + Speaker + AGC |
|---|---|---|---|
| SI-SDR (dB) | 10.38 | 11.80 | 11.62 |
| PESQ | 2.88 | 2.98 | 3.01 |
| WER (%) | 15.86 | 14.42 | 13.20 |
The model improved the Signal-to-Distortion Ratio (SI-SDR) by 1.42 dB. Most notably, the Word Error Rate (WER) decreased from 15.86% to 13.20% when using the full configuration, representing a nearly 17% relative improvement.
Conclusion
NeuralEcho offers a highly efficient and effective solution for unified audio enhancement. By explicitly calculating second-order statistics and employing joint training for AGC, it outperforms larger models while remaining suitable for edge device deployment. Future work will explore extending this framework to multi-microphone beamforming arrays and further optimizing real-time latency for communication applications.
Summary updated